-- Logs begin at Wed 2022-06-15 09:50:27 UTC, end at Wed 2022-06-15 10:51:10 UTC. -- Jun 15 10:17:11.133271 dv-cinder-843493-2-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Jun 15 10:17:11.141822 dv-cinder-843493-2-iscsi devstack@keystone.service[80111]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Jun 15 10:17:11.144536 dv-cinder-843493-2-iscsi devstack@keystone.service[80111]: *** Starting uWSGI 2.0.18-debian (64bit) on [Wed Jun 15 10:17:11 2022] *** Jun 15 10:17:11.144536 dv-cinder-843493-2-iscsi devstack@keystone.service[80111]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Jun 15 10:17:11.144536 dv-cinder-843493-2-iscsi devstack@keystone.service[80111]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Jun 15 10:17:11.144536 dv-cinder-843493-2-iscsi devstack@keystone.service[80111]: nodename: dv-cinder-843493-2-iscsi Jun 15 10:17:11.144536 dv-cinder-843493-2-iscsi devstack@keystone.service[80111]: machine: x86_64 Jun 15 10:17:11.144536 dv-cinder-843493-2-iscsi devstack@keystone.service[80111]: clock source: unix Jun 15 10:17:11.144536 dv-cinder-843493-2-iscsi devstack@keystone.service[80111]: pcre jit disabled Jun 15 10:17:11.144536 dv-cinder-843493-2-iscsi devstack@keystone.service[80111]: detected number of CPU cores: 4 Jun 15 10:17:11.144536 dv-cinder-843493-2-iscsi devstack@keystone.service[80111]: current working directory: / Jun 15 10:17:11.144536 dv-cinder-843493-2-iscsi devstack@keystone.service[80111]: detected binary path: /usr/bin/uwsgi-core Jun 15 10:17:11.145071 dv-cinder-843493-2-iscsi devstack@keystone.service[80111]: your processes number limit is 47680 Jun 15 10:17:11.145071 dv-cinder-843493-2-iscsi devstack@keystone.service[80111]: your memory page size is 4096 bytes Jun 15 10:17:11.145071 dv-cinder-843493-2-iscsi devstack@keystone.service[80111]: detected max file descriptor number: 2048 Jun 15 10:17:11.145071 dv-cinder-843493-2-iscsi devstack@keystone.service[80111]: lock engine: pthread robust mutexes Jun 15 10:17:11.145071 dv-cinder-843493-2-iscsi devstack@keystone.service[80111]: thunder lock: enabled Jun 15 10:17:11.145071 dv-cinder-843493-2-iscsi devstack@keystone.service[80111]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Jun 15 10:17:11.145071 dv-cinder-843493-2-iscsi devstack@keystone.service[80111]: Python version: 3.8.10 (default, Mar 15 2022, 12:22:08) [GCC 9.4.0] Jun 15 10:17:11.188385 dv-cinder-843493-2-iscsi devstack@keystone.service[80111]: Python main interpreter initialized at 0x55f4df3da9c0 Jun 15 10:17:11.188385 dv-cinder-843493-2-iscsi devstack@keystone.service[80111]: python threads support enabled Jun 15 10:17:11.188385 dv-cinder-843493-2-iscsi devstack@keystone.service[80111]: your server socket listen backlog is limited to 100 connections Jun 15 10:17:11.188385 dv-cinder-843493-2-iscsi devstack@keystone.service[80111]: your mercy for graceful operations on workers is 90 seconds Jun 15 10:17:11.188724 dv-cinder-843493-2-iscsi devstack@keystone.service[80111]: mapped 403077 bytes (393 KB) for 2 cores Jun 15 10:17:11.188801 dv-cinder-843493-2-iscsi devstack@keystone.service[80111]: *** Operational MODE: preforking *** Jun 15 10:17:11.189058 dv-cinder-843493-2-iscsi devstack@keystone.service[80111]: *** uWSGI is running in multiple interpreter mode *** Jun 15 10:17:11.189058 dv-cinder-843493-2-iscsi devstack@keystone.service[80111]: spawned uWSGI master process (pid: 80111) Jun 15 10:17:11.189258 dv-cinder-843493-2-iscsi systemd[1]: Started Devstack devstack@keystone.service. Jun 15 10:17:11.190496 dv-cinder-843493-2-iscsi devstack@keystone.service[80111]: spawned uWSGI worker 1 (pid: 80119, cores: 1) Jun 15 10:17:11.190496 dv-cinder-843493-2-iscsi devstack@keystone.service[80111]: spawned uWSGI worker 2 (pid: 80120, cores: 1) Jun 15 10:17:11.190496 dv-cinder-843493-2-iscsi devstack@keystone.service[80111]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Jun 15 10:17:16.704724 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 15 10:17:16.704724 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: from cryptography.utils import int_from_bytes Jun 15 10:17:16.705514 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 15 10:17:16.705514 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: from cryptography.utils import int_from_bytes Jun 15 10:17:16.705922 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 15 10:17:16.705922 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: from cryptography.utils import int_from_bytes Jun 15 10:17:16.706022 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 15 10:17:16.706022 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: from cryptography.utils import int_from_bytes Jun 15 10:17:16.754405 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jun 15 10:17:16.754593 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jun 15 10:17:16.754784 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] command line args: [] {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jun 15 10:17:16.754911 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jun 15 10:17:16.755050 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jun 15 10:17:16.755157 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jun 15 10:17:16.755239 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] ================================================================================ {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jun 15 10:17:16.755317 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG uwsgi [-] command line args: [] {{(pid=80119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jun 15 10:17:16.755497 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] admin_token = **** {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 10:17:16.755581 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jun 15 10:17:16.755738 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG uwsgi [-] ================================================================================ {{(pid=80119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jun 15 10:17:16.755824 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] config_dir = [] {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 10:17:16.755905 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 10:17:16.755997 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG uwsgi [-] admin_token = **** {{(pid=80119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 10:17:16.756073 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] config_source = [] {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 10:17:16.756243 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG uwsgi [-] config_dir = [] {{(pid=80119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 10:17:16.756326 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] debug = True {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 10:17:16.756435 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 10:17:16.756513 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 10:17:16.756690 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG uwsgi [-] config_source = [] {{(pid=80119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 10:17:16.756814 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG uwsgi [-] debug = True {{(pid=80119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 10:17:16.756930 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 10:17:16.757053 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 10:17:16.757172 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 10:17:16.757317 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 10:17:16.757436 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 10:17:16.757436 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 10:17:16.757657 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 10:17:16.757776 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 10:17:16.757894 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] list_limit = None {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 10:17:16.757894 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] log_config_append = None {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 10:17:16.758111 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 10:17:16.758232 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 10:17:16.758356 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG uwsgi [-] list_limit = None {{(pid=80119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 10:17:16.758475 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] log_dir = None {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 10:17:16.758594 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG uwsgi [-] log_config_append = None {{(pid=80119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 10:17:16.758713 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] log_file = None {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 10:17:16.758825 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 10:17:16.758942 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 10:17:16.759066 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG uwsgi [-] log_dir = None {{(pid=80119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 10:17:16.759066 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG uwsgi [-] log_file = None {{(pid=80119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 10:17:16.759298 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 10:17:16.759410 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 10:17:16.759512 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 10:17:16.759617 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 10:17:16.759730 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 10:17:16.759730 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 10:17:16.759952 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 10:17:16.760060 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 10:17:16.760182 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 10:17:16.760308 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 10:17:16.760386 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 10:17:16.760386 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 10:17:16.760572 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 10:17:16.760688 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 10:17:16.760763 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 10:17:16.760763 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 10:17:16.760945 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 10:17:16.761059 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 10:17:16.761140 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 10:17:16.761216 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 10:17:16.761334 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 10:17:16.761422 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 10:17:16.761493 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 10:17:16.761493 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 10:17:16.761687 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 10:17:16.761766 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 10:17:16.761881 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 10:17:16.761968 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 10:17:16.762133 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.49/identity {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 10:17:16.762213 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 10:17:16.762327 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] publish_errors = False {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 10:17:16.762536 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 10:17:16.762619 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.49/identity {{(pid=80119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 10:17:16.762764 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 10:17:16.762840 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG uwsgi [-] publish_errors = False {{(pid=80119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 10:17:16.762942 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 10:17:16.763019 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 10:17:16.763212 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 10:17:16.763301 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 10:17:16.763418 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 10:17:16.763510 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 10:17:16.763653 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 10:17:16.763771 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] standard_threads = False {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 10:17:16.763890 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 10:17:16.763967 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 10:17:16.764128 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG uwsgi [-] standard_threads = False {{(pid=80119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 10:17:16.764260 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 10:17:16.764334 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 10:17:16.764405 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 10:17:16.764469 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] use_journal = False {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 10:17:16.764596 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 10:17:16.764672 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] use_json = False {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 10:17:16.764775 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 10:17:16.764858 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] use_stderr = False {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 10:17:16.764951 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] use_syslog = False {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 10:17:16.765027 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG uwsgi [-] use_journal = False {{(pid=80119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 10:17:16.765127 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG uwsgi [-] use_json = False {{(pid=80119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 10:17:16.765213 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 10:17:16.765290 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG uwsgi [-] use_stderr = False {{(pid=80119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 10:17:16.765423 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.765500 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG uwsgi [-] use_syslog = False {{(pid=80119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 10:17:16.765663 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.765741 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 10:17:16.765955 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.766033 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.766199 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.766279 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.766447 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.766523 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.766684 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.766762 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.766932 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.767009 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.767193 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.767310 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.767441 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.767519 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.767677 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.767759 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.767923 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.767999 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.768155 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.768233 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.768399 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.768475 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.768632 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.768708 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.768877 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.768996 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.769095 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.769188 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.769346 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.769447 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.769629 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.769726 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.769828 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.769924 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.770070 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.770186 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.770294 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.770395 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.770546 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.770650 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.770782 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.770884 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.771038 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.771173 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.771311 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.771413 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.771541 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.771654 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.771796 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.771915 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.772030 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.772149 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.772264 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.772377 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.772498 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.772612 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.772731 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.772850 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.772964 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.773088 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.773203 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.773317 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.773429 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.773542 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.773654 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.773775 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.773889 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.774002 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.774114 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.774227 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.774340 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.774461 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.774580 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.774693 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.774805 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.774917 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.775029 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.775240 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.775319 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.775442 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.775519 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.775589 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.775659 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.775737 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.775820 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.775898 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.776028 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.776112 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.776257 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.776352 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.776492 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.776575 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.776721 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.776804 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.776955 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.777040 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.777187 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.777293 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.777422 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.777526 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.777649 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.777759 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.777885 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.777994 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.778100 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.778201 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.778351 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.778439 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.778557 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.778631 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.778782 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.778869 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.779012 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.779126 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.779280 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.779353 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.779513 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.779589 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.779746 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.779831 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.779975 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.780062 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.780212 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.780285 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.780444 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.780530 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.780672 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.780757 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.780907 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.780985 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.781142 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.781216 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.781406 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.782163 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.782244 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.782408 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.782491 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.782647 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.782724 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.782883 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.782962 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.783129 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.783221 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.783368 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.783454 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.783616 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.783694 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.783853 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.783930 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.784089 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.784173 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.784330 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.784409 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.784567 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.784643 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.784803 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.784887 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.785037 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.785114 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.785272 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.785355 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.785508 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.785594 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.785742 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.785818 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.785980 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.786058 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.786218 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.786302 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.786454 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.786532 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.786693 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.786776 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.786930 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.787137 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.787217 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.787287 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.787421 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.787506 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.787648 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.787736 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.787877 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.787967 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.788106 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.788184 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.788339 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.788422 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.788565 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.788651 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.788808 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.788922 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.789037 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.789254 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.789497 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.790588 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.790868 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.791181 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.792182 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.792342 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.792461 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.793100 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.793231 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.793352 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.793468 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.794075 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] auth.external = None {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.794075 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.794212 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.794212 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.794323 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.794428 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG uwsgi [-] auth.external = None {{(pid=80119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.794508 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.794609 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.794807 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] auth.password = None {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.794807 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] auth.token = None {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.794951 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.795023 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.795108 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.795182 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.795265 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG uwsgi [-] auth.password = None {{(pid=80119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.795265 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG uwsgi [-] auth.token = None {{(pid=80119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.795396 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.795476 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.795549 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.795619 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.795698 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.795771 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.795842 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.795912 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.795983 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.796054 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.796132 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] credential.caching = True {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.796204 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.796275 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.796345 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.796421 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.796494 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG uwsgi [-] credential.caching = True {{(pid=80119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.796572 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.796645 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.796715 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.796793 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.796871 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.796943 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.797021 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.797093 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.797244 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.797842 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.797920 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.798048 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.798168 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.798293 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.798440 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.798673 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.798789 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.798932 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.799059 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.799241 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.799340 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.799483 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.799609 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.799750 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.799842 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.799998 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.800108 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.800259 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.800338 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] federation.caching = True {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.800504 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG uwsgi [-] federation.caching = True {{(pid=80119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.800592 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.800745 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.800824 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.800997 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.801075 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.801248 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.801340 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.801484 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.801569 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.801726 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.801804 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.801964 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.802049 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.802208 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.802286 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.802448 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.802525 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.802689 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.802779 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.802934 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.803012 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.803193 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] identity.caching = True {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.803274 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.803434 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG uwsgi [-] identity.caching = True {{(pid=80119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.803596 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.803674 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.803745 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.803908 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.803988 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.804141 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.804221 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.804640 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.804720 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.804871 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.804954 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.805108 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.805188 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.805346 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.805462 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.805595 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.805700 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.805828 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.805935 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.806058 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.806163 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.806286 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.806368 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.806516 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.806595 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.806761 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.806839 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.807007 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.807136 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.807279 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.807359 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.807524 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.807602 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.807773 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.807850 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.808018 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.808098 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.808255 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.808333 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.808487 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.808566 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.808719 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.808798 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.808945 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.809022 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.809177 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.809254 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.809413 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.809491 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.809650 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.809728 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.809888 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.809965 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.810118 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.810196 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.810362 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.810439 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.810605 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.810683 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.810840 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.810921 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.811075 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.811170 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.811343 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.811422 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.811573 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.811652 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.811809 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.811888 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.812041 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.812120 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.812273 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.812352 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.812508 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.812586 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.812768 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.812863 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.813010 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.813091 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.813247 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.813332 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.813485 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.813563 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.813736 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.813817 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.813964 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.814043 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.814195 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.814274 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.814426 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.814505 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.814675 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.814755 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.814908 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.814986 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.815168 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.815250 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.815403 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.815482 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.815632 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] ldap.user = None {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.815711 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.815862 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG uwsgi [-] ldap.user = None {{(pid=80119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.815941 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.816100 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.816179 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.816335 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.816413 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.816581 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.816661 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.816827 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.816903 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.817070 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.817156 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.817307 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.817387 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.817537 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.817621 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.817766 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.817851 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.817997 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.818075 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.818230 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.818308 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.818459 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.818545 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.818690 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.818767 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.818938 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.819016 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.819205 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.819302 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.819453 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.819532 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.819700 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.819778 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.819928 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.820014 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.820169 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.820247 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.820404 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.820483 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.820640 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.820727 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.820876 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.820955 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.821112 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.821191 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.821352 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.821438 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.821598 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.821676 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.821834 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.821913 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.822074 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.822160 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.822303 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.822381 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.822545 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.822625 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.822781 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.822868 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.823014 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.823111 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.823274 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.823352 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.823514 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.823606 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.823760 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.823838 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.823990 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.824069 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.824222 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.824307 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.824461 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.824540 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] resource.caching = True {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.824704 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG uwsgi [-] resource.caching = True {{(pid=80119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.824788 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.824948 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.825035 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.825182 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.825262 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.825419 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.825496 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.825669 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.825756 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.825906 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.825985 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.826147 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.826224 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.826381 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.826533 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.826609 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.826681 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.826840 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] role.caching = True {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.826920 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.827075 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG uwsgi [-] role.caching = True {{(pid=80119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.827208 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] role.driver = sql {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.827326 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.827404 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG uwsgi [-] role.driver = sql {{(pid=80119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.827561 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.827639 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.827810 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.827898 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.828058 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.828136 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.828303 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.828400 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.828576 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.828666 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.828824 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.828904 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.829068 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.829146 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.829314 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.829393 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.829541 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.829618 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.829781 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.829858 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.830025 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.830104 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.830271 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.830349 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.830516 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.830595 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.830762 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.830943 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.831205 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.831289 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.831445 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.831523 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.831690 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.831773 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.831936 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.832015 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.832181 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.832260 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.832412 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.832493 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.832652 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.832729 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.832902 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.832980 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.833141 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.833261 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.833372 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.833451 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.833606 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.833691 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.833843 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.833922 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.834078 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.834157 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.834332 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.834419 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.834575 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.834653 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.834811 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.834890 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.835045 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.835188 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.835298 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.835377 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.835527 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.835605 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] token.caching = True {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.835760 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.835847 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG uwsgi [-] token.caching = True {{(pid=80119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.835996 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.836074 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.836228 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.836306 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.836469 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.836554 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.836717 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.836795 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.836953 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.837031 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.837189 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.837275 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.837428 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.837506 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.837669 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.837746 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.837902 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.837987 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.838138 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.838218 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.838371 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.838449 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.838605 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.838690 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.838850 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.838928 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.839078 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.839175 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.839340 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.839431 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.839582 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.839660 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.839879 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.839957 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] database.connection = **** {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.840123 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.840210 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG uwsgi [-] database.connection = **** {{(pid=80119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.840371 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.840460 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.840609 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.840689 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.840849 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.840926 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.841081 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.841159 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.841319 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.841397 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.841553 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.841631 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.841788 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.841866 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.842026 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.842104 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.842259 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.842336 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.842500 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.842578 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.842731 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.842811 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.842969 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.843047 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.843232 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.843311 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.843469 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.843548 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.843703 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.843781 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.843938 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.844534 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.844704 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.844784 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.844857 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jun 15 10:17:16.844927 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:17:16.845103 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80119) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jun 15 10:17:16.961548 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80120) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 10:17:16.961736 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80119) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 10:17:16.961824 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80120) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 10:17:16.961902 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80120) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 10:17:16.961988 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80119) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 10:17:16.962137 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80119) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 10:17:16.970894 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80120) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 10:17:16.971477 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80119) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 10:17:16.998614 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80120) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 10:17:16.998843 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80120) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 10:17:16.999317 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80119) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 10:17:16.999556 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80119) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 10:17:17.052499 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80120) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 10:17:17.052974 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80119) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 10:17:17.101571 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80120) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 10:17:17.101803 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80120) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 10:17:17.102467 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80119) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 10:17:17.102710 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80119) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 10:17:17.168082 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80119) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 10:17:17.168291 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80120) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 10:17:17.168383 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80119) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 10:17:17.168521 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80119) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 10:17:17.168625 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80120) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 10:17:17.168741 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80120) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 10:17:17.168823 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80119) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 10:17:17.168956 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80119) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 10:17:17.169037 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80120) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 10:17:17.169168 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80120) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 10:17:17.169251 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80119) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 10:17:17.169383 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80120) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 10:17:17.193787 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80119) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 10:17:17.193920 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80120) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 10:17:17.194023 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80119) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 10:17:17.194094 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80120) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 10:17:17.194244 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80119) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 10:17:17.194362 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80120) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 10:17:17.194446 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80119) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 10:17:17.194536 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80120) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 10:17:17.194671 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80119) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 10:17:17.194759 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80120) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 10:17:17.194888 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80120) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 10:17:17.194971 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80119) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 10:17:17.195110 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80120) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 10:17:17.195213 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80119) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 10:17:17.206110 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80120) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 10:17:17.206579 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80119) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 10:17:17.218838 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80120) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 10:17:17.219071 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80119) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 10:17:17.219202 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80120) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 10:17:17.219308 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80119) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 10:17:17.219392 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80120) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 10:17:17.219530 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80120) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 10:17:17.219611 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80119) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 10:17:17.219745 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80120) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 10:17:17.219828 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80119) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 10:17:17.219956 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80120) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 10:17:17.220039 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80119) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 10:17:17.220166 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80120) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 10:17:17.220249 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80119) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 10:17:17.220379 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80119) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 10:17:17.237836 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80120) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 10:17:17.238067 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80120) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 15 10:17:17.238407 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80120) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 10:17:17.238788 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80120) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 15 10:17:17.239058 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80120) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 10:17:17.239346 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80120) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 10:17:17.239603 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80120) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 10:17:17.239860 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80120) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 10:17:17.240266 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80119) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 10:17:17.240479 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80119) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 15 10:17:17.240815 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80119) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 10:17:17.241018 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80119) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 15 10:17:17.241480 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80119) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 10:17:17.241746 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80119) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 10:17:17.242011 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80119) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 10:17:17.242271 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80119) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 10:17:17.253952 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80119) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 10:17:17.254232 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80119) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 10:17:17.254501 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80119) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 10:17:17.254586 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80120) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 10:17:17.254787 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80119) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 10:17:17.254889 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80120) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 10:17:17.255124 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80120) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 10:17:17.255404 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80120) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 10:17:17.263994 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80120) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 15 10:17:17.264434 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80119) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 15 10:17:17.267365 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80120) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 15 10:17:17.267646 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80120) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 10:17:17.267888 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80119) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 15 10:17:17.267972 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80120) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 10:17:17.268172 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80119) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 10:17:17.268270 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80120) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 10:17:17.268445 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80120) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 10:17:17.268527 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80119) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 10:17:17.268699 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80120) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 10:17:17.268782 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80119) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 10:17:17.268965 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80120) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 10:17:17.269050 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80119) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 10:17:17.269236 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80120) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 10:17:17.269317 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80119) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 10:17:17.269502 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80120) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 10:17:17.269588 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80119) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 10:17:17.269767 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80120) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 10:17:17.269850 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80119) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 10:17:17.270029 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80120) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 10:17:17.270110 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80119) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 10:17:17.270296 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80119) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 10:17:17.270563 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80119) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 10:17:17.298463 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80120) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 10:17:17.300126 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80120) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 15 10:17:17.300220 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80119) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 10:17:17.300403 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80120) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 10:17:17.301779 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80119) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 15 10:17:17.302060 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80119) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 10:17:17.305891 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80120) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 15 10:17:17.306169 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80120) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 10:17:17.306453 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80120) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 10:17:17.310558 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80119) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 15 10:17:17.311015 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80119) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 10:17:17.311541 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80119) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 10:17:17.314495 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80120) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 15 10:17:17.314775 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80120) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 10:17:17.319265 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80120) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 15 10:17:17.319553 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80120) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 10:17:17.319831 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80120) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 10:17:17.320107 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80120) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 10:17:17.320377 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80120) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 10:17:17.320647 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80120) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 10:17:17.320913 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80120) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 10:17:17.321177 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80120) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 10:17:17.325361 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80119) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 15 10:17:17.325869 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80119) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 10:17:17.331991 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80119) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 15 10:17:17.332284 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80119) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 10:17:17.332566 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80119) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 10:17:17.332846 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80119) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 10:17:17.333116 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80119) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 10:17:17.333387 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80119) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 10:17:17.333656 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80119) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 10:17:17.333925 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80119) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 10:17:17.339929 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80120) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 10:17:17.340210 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80120) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 10:17:17.345307 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80120) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 15 10:17:17.348969 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80120) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 10:17:17.349266 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80120) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 10:17:17.353149 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80119) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 10:17:17.353432 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80119) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 10:17:17.354366 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80120) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 15 10:17:17.358145 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80119) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 15 10:17:17.358310 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80120) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 15 10:17:17.362391 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80119) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 10:17:17.362691 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80119) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 10:17:17.362774 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80120) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 10:17:17.363036 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80120) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 10:17:17.363354 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80120) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 10:17:17.363635 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80120) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 10:17:17.363908 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80120) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 10:17:17.364183 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80120) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 10:17:17.370570 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80119) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 15 10:17:17.372318 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80119) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 15 10:17:17.376214 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80119) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 10:17:17.376515 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80119) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 10:17:17.376803 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80119) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 10:17:17.377085 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80119) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 10:17:17.377362 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80119) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 10:17:17.377640 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80119) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 10:17:17.383573 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80120) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 15 10:17:17.383865 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80120) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 10:17:17.384129 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80120) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 10:17:17.384398 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80120) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 10:17:17.392338 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80120) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 10:17:17.394029 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80120) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 10:17:17.394334 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80120) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 10:17:17.397494 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80119) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 15 10:17:17.397877 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80119) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 10:17:17.398240 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80119) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 10:17:17.398507 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80119) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 10:17:17.399344 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80120) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 15 10:17:17.399634 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80120) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 10:17:17.399922 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80120) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 10:17:17.400228 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80120) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 10:17:17.400850 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80120) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 10:17:17.409225 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80119) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 10:17:17.410941 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80119) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 10:17:17.411257 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80119) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 10:17:17.414307 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80119) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 15 10:17:17.414597 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80119) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 10:17:17.415398 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80120) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 15 10:17:17.415683 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80120) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 10:17:17.415964 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80120) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 10:17:17.416240 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80120) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 10:17:17.416385 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80119) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 10:17:17.416522 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80120) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 10:17:17.416687 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80119) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 10:17:17.416792 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80120) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 10:17:17.416986 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80119) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 10:17:17.417076 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80120) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 10:17:17.431514 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80119) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 15 10:17:17.431805 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80119) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 10:17:17.432121 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80119) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 10:17:17.432404 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80119) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 10:17:17.432684 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80119) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 10:17:17.432958 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80119) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 10:17:17.434396 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80119) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 10:17:17.436371 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80120) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 15 10:17:17.440210 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80120) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 15 10:17:17.443509 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80120) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 10:17:17.445163 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80120) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 10:17:17.446946 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80120) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 15 10:17:17.447243 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80120) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 10:17:17.447522 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80120) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 10:17:17.455460 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80120) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 10:17:17.455922 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80119) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 15 10:17:17.457087 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80120) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 15 10:17:17.459221 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80119) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 15 10:17:17.460320 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80120) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 10:17:17.460654 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80120) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 10:17:17.461016 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80120) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 10:17:17.461315 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80120) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 10:17:17.464696 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80119) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 10:17:17.466296 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80119) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 10:17:17.468106 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80119) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 15 10:17:17.468387 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80119) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 10:17:17.468668 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80119) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 10:17:17.472402 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80120) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 15 10:17:17.472643 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80120) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 10:17:17.472920 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80120) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 10:17:17.476456 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80119) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 10:17:17.478075 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80119) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 15 10:17:17.480788 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80120) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 15 10:17:17.481070 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80120) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 10:17:17.481481 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80119) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 10:17:17.481699 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80120) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 10:17:17.481902 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80119) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 10:17:17.482032 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80120) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 10:17:17.482189 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80119) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 10:17:17.482398 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80120) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 10:17:17.482598 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80119) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 10:17:17.482807 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80120) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 10:17:17.482807 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80120) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 10:17:17.483197 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80120) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 10:17:17.483466 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80120) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 10:17:17.483741 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80120) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 10:17:17.484010 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80120) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 10:17:17.484276 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80120) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 10:17:17.484543 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80120) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 10:17:17.484859 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80120) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 10:17:17.492225 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80119) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 15 10:17:17.492518 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80119) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 10:17:17.492799 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80119) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 10:17:17.505849 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80119) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 15 10:17:17.506136 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80119) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 10:17:17.506418 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80119) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 10:17:17.506693 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80119) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 10:17:17.506967 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80119) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 10:17:17.507292 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80119) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 10:17:17.507576 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80119) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 10:17:17.507851 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80119) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 10:17:17.508215 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80119) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 10:17:17.508594 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80119) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 10:17:17.509569 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80119) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 10:17:17.509949 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80119) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 10:17:17.510248 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80119) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 10:17:17.510549 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80119) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 10:17:17.525556 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x55f4df3da9c0 pid: 80120 (default app) Jun 15 10:17:17.529051 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6c0051-bb07-45e9-839e-814281a60e1f None None] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:17:17.529354 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6c0051-bb07-45e9-839e-814281a60e1f None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:17:17.529632 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6c0051-bb07-45e9-839e-814281a60e1f None None] PATH_INFO: `/v3` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:17:17.535103 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1/1] 10.222.0.49 () {50 vars in 791 bytes} [Wed Jun 15 10:17:17 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:17:17.550213 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x55f4df3da9c0 pid: 80119 (default app) Jun 15 10:17:25.472136 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de3e54b-6b9a-4a46-a1d9-e81986450278 None None] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:17:25.472697 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de3e54b-6b9a-4a46-a1d9-e81986450278 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:17:25.472697 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de3e54b-6b9a-4a46-a1d9-e81986450278 None None] PATH_INFO: `/` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:17:25.474054 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 2/2] 10.222.0.49 () {54 vars in 926 bytes} [Wed Jun 15 10:17:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 15 10:17:25.481525 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec743f9-c177-4ed5-8a75-d142e63252a1 None None] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:17:25.481939 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec743f9-c177-4ed5-8a75-d142e63252a1 None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:17:25.482092 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec743f9-c177-4ed5-8a75-d142e63252a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:17:25.552528 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: INFO dbcounter [None req-1ec743f9-c177-4ed5-8a75-d142e63252a1 None None] Registered counter for database keystone Jun 15 10:17:25.574069 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG oslo_db.sqlalchemy.engines [None req-1ec743f9-c177-4ed5-8a75-d142e63252a1 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80119) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jun 15 10:17:25.580456 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writer thread running {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jun 15 10:17:25.758291 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG passlib.handlers.bcrypt [None req-1ec743f9-c177-4ed5-8a75-d142e63252a1 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80119) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jun 15 10:17:25.758641 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG passlib.handlers.bcrypt [None req-1ec743f9-c177-4ed5-8a75-d142e63252a1 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80119) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jun 15 10:17:25.813961 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.auth.core [None req-1ec743f9-c177-4ed5-8a75-d142e63252a1 None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:17:25.815595 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-1ec743f9-c177-4ed5-8a75-d142e63252a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:17:25.936057 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1/3] 10.222.0.49 () {58 vars in 1023 bytes} [Wed Jun 15 10:17:25 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 458 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Jun 15 10:17:26.121178 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de3e54b-6b9a-4a46-a1d9-e81986450278 None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:17:26.195885 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: INFO dbcounter [None req-5de3e54b-6b9a-4a46-a1d9-e81986450278 None None] Registered counter for database keystone Jun 15 10:17:26.215517 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG oslo_db.sqlalchemy.engines [None req-5de3e54b-6b9a-4a46-a1d9-e81986450278 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80120) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jun 15 10:17:26.222335 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writer thread running {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jun 15 10:17:26.344131 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de3e54b-6b9a-4a46-a1d9-e81986450278 None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:17:26.348436 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e83f7ab-695c-4c37-8277-6818c4507281 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:17:26.349103 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e83f7ab-695c-4c37-8277-6818c4507281 admin admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:17:26.349397 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e83f7ab-695c-4c37-8277-6818c4507281 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:17:26.349679 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e83f7ab-695c-4c37-8277-6818c4507281 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:17:26.357829 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e83f7ab-695c-4c37-8277-6818c4507281 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:17:26.409429 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: WARNING keystone.common.rbac_enforcer.enforcer [None req-8e83f7ab-695c-4c37-8277-6818c4507281 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jun 15 10:17:26.409723 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e83f7ab-695c-4c37-8277-6818c4507281 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:17:26.419303 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 3/4] 10.222.0.49 () {58 vars in 1324 bytes} [Wed Jun 15 10:17:26 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 15 10:17:26.425174 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec743f9-c177-4ed5-8a75-d142e63252a1 None None] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:17:26.442568 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec743f9-c177-4ed5-8a75-d142e63252a1 None None] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:17:26.446933 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5e60c7-4bfd-4ddd-97d6-ce3f35730b87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:17:26.448097 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5e60c7-4bfd-4ddd-97d6-ce3f35730b87 admin admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:17:26.448387 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5e60c7-4bfd-4ddd-97d6-ce3f35730b87 admin admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:17:26.448661 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5e60c7-4bfd-4ddd-97d6-ce3f35730b87 admin admin] PATH_INFO: `/v3/projects/c8da8e1de64f469c9cc5f9a5f416060d` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:17:26.450169 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e5e60c7-4bfd-4ddd-97d6-ce3f35730b87 admin admin] RBAC: Authorizing `identity:get_project(project_id=c8da8e1de64f469c9cc5f9a5f416060d)` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:17:26.500598 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: WARNING keystone.common.rbac_enforcer.enforcer [None req-8e5e60c7-4bfd-4ddd-97d6-ce3f35730b87 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jun 15 10:17:26.501142 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e5e60c7-4bfd-4ddd-97d6-ce3f35730b87 admin admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:17:26.505515 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 2/5] 10.222.0.49 () {56 vars in 1207 bytes} [Wed Jun 15 10:17:26 2022] GET /identity/v3/projects/c8da8e1de64f469c9cc5f9a5f416060d => generated 344 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:17:28.592021 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1687133f-9be1-4141-9fce-788036e5e577 None None] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:17:28.592560 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1687133f-9be1-4141-9fce-788036e5e577 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:17:28.592560 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1687133f-9be1-4141-9fce-788036e5e577 None None] PATH_INFO: `/` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:17:28.597287 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 4/6] 10.222.0.49 () {54 vars in 926 bytes} [Wed Jun 15 10:17:28 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:17:28.602632 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda9c5e7-4059-45e5-bd5e-023092ffda5e None None] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:17:28.603057 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda9c5e7-4059-45e5-bd5e-023092ffda5e None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:17:28.603236 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda9c5e7-4059-45e5-bd5e-023092ffda5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:17:28.622152 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.auth.core [None req-eda9c5e7-4059-45e5-bd5e-023092ffda5e None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:17:28.623692 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-eda9c5e7-4059-45e5-bd5e-023092ffda5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:17:28.631525 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 3/7] 10.222.0.49 () {58 vars in 1023 bytes} [Wed Jun 15 10:17:28 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Jun 15 10:17:28.653710 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1687133f-9be1-4141-9fce-788036e5e577 None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:17:28.666321 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1687133f-9be1-4141-9fce-788036e5e577 None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:17:28.670593 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d60324-5d08-40b7-a6cb-79b598d134ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:17:28.671255 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d60324-5d08-40b7-a6cb-79b598d134ed admin admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:17:28.671547 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d60324-5d08-40b7-a6cb-79b598d134ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:17:28.671885 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d60324-5d08-40b7-a6cb-79b598d134ed admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:17:28.680361 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d60324-5d08-40b7-a6cb-79b598d134ed admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:17:28.682139 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d60324-5d08-40b7-a6cb-79b598d134ed admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:17:28.690928 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 5/8] 10.222.0.49 () {58 vars in 1324 bytes} [Wed Jun 15 10:17:28 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 15 10:17:28.695984 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda9c5e7-4059-45e5-bd5e-023092ffda5e None None] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:17:28.704807 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda9c5e7-4059-45e5-bd5e-023092ffda5e None None] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:17:28.709146 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ca6a72-d6fb-410b-805e-e563a8e332ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:17:28.710001 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ca6a72-d6fb-410b-805e-e563a8e332ab admin admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:17:28.710287 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ca6a72-d6fb-410b-805e-e563a8e332ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:17:28.710563 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ca6a72-d6fb-410b-805e-e563a8e332ab admin admin] PATH_INFO: `/v3/users/fcdb2d8e71bd4fe6b9b1431183facba7` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:17:28.711949 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ca6a72-d6fb-410b-805e-e563a8e332ab admin admin] RBAC: Authorizing `identity:get_user(user_id=fcdb2d8e71bd4fe6b9b1431183facba7)` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:17:28.713767 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ca6a72-d6fb-410b-805e-e563a8e332ab admin admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:17:28.715342 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 4/9] 10.222.0.49 () {56 vars in 1198 bytes} [Wed Jun 15 10:17:28 2022] GET /identity/v3/users/fcdb2d8e71bd4fe6b9b1431183facba7 => generated 247 bytes in 20 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:17:30.915674 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7954b636-4fc2-4079-9fba-8791d5f4cc70 None None] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:17:30.916214 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7954b636-4fc2-4079-9fba-8791d5f4cc70 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:17:30.916214 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7954b636-4fc2-4079-9fba-8791d5f4cc70 None None] PATH_INFO: `/` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:17:30.917589 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 6/10] 10.222.0.49 () {54 vars in 926 bytes} [Wed Jun 15 10:17:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:17:30.923901 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306b5dc4-e1c8-456f-9c75-a68aeac1dc34 None None] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:17:30.924401 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306b5dc4-e1c8-456f-9c75-a68aeac1dc34 None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:17:30.924486 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306b5dc4-e1c8-456f-9c75-a68aeac1dc34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:17:30.941025 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.auth.core [None req-306b5dc4-e1c8-456f-9c75-a68aeac1dc34 None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:17:30.942203 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-306b5dc4-e1c8-456f-9c75-a68aeac1dc34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:17:30.957888 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c74124-8584-49cf-b9d9-991332f15ee6 None None] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:17:30.958175 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c74124-8584-49cf-b9d9-991332f15ee6 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:17:30.958452 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c74124-8584-49cf-b9d9-991332f15ee6 None None] PATH_INFO: `/` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:17:30.959844 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 7/11] 10.222.0.49 () {54 vars in 926 bytes} [Wed Jun 15 10:17:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 15 10:17:30.964879 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62188b57-9e37-4ce8-b531-f12239f8179b None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:17:30.965171 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62188b57-9e37-4ce8-b531-f12239f8179b None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:17:30.965461 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62188b57-9e37-4ce8-b531-f12239f8179b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:17:31.005465 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG passlib.handlers.bcrypt [None req-62188b57-9e37-4ce8-b531-f12239f8179b None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80120) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jun 15 10:17:31.005880 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG passlib.handlers.bcrypt [None req-62188b57-9e37-4ce8-b531-f12239f8179b None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80120) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jun 15 10:17:31.021434 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 5/12] 10.222.0.49 () {58 vars in 1023 bytes} [Wed Jun 15 10:17:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 15 10:17:31.044615 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306b5dc4-e1c8-456f-9c75-a68aeac1dc34 None None] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:17:31.052130 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-62188b57-9e37-4ce8-b531-f12239f8179b None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:17:31.053993 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-62188b57-9e37-4ce8-b531-f12239f8179b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:17:31.067763 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=100 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:17:31.147427 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306b5dc4-e1c8-456f-9c75-a68aeac1dc34 None None] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:17:31.152917 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 8/13] 10.222.0.49 () {58 vars in 1023 bytes} [Wed Jun 15 10:17:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 15 10:17:31.175656 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62188b57-9e37-4ce8-b531-f12239f8179b None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:17:31.181866 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f3409d-9c7e-4b67-9b3b-3add3e2d38ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:17:31.182672 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f3409d-9c7e-4b67-9b3b-3add3e2d38ab None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:17:31.182959 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f3409d-9c7e-4b67-9b3b-3add3e2d38ab None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:17:31.183270 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f3409d-9c7e-4b67-9b3b-3add3e2d38ab None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:17:31.194752 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f3409d-9c7e-4b67-9b3b-3add3e2d38ab None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 10:17:31.206500 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application [None req-44f3409d-9c7e-4b67-9b3b-3add3e2d38ab None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 10:17:31.206500 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:17:31.206500 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:17:31.206500 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:17:31.206500 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:17:31.206500 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:17:31.206500 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:17:31.206500 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:17:31.206500 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:17:31.206500 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:17:31.206500 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:17:31.206500 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:17:31.206500 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 10:17:31.206500 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 10:17:31.206500 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 10:17:31.206500 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 10:17:31.207670 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 10:17:31.207670 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 10:17:31.207670 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 10:17:31.207670 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 10:17:31.207670 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:17:31.207670 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:17:31.207670 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:17:31.207670 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:17:31.207670 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:17:31.207670 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:17:31.207670 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:17:31.207670 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application with Lock( Jun 15 10:17:31.207670 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:17:31.207670 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:17:31.207670 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:17:31.207670 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:17:31.213228 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:17:31.213228 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:17:31.213228 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:17:31.213228 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:17:31.213228 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 10:17:31.213228 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 10:17:31.213228 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 10:17:31.213228 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 10:17:31.213228 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 10:17:31.213228 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 10:17:31.213228 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 10:17:31.213228 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application Jun 15 10:17:31.213228 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 6/14] 10.222.0.49 () {56 vars in 1096 bytes} [Wed Jun 15 10:17:31 2022] GET /identity/v3/roles/admin => generated 83 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 10:17:31.213898 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f3409d-9c7e-4b67-9b3b-3add3e2d38ab None admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:17:31.276903 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62188b57-9e37-4ce8-b531-f12239f8179b None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:17:31.279235 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:17:31.295684 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=100 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:17:31.313558 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f3409d-9c7e-4b67-9b3b-3add3e2d38ab None admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:17:31.316524 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b79614-1783-46df-8ae6-11b9d1f2a38c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:17:31.317461 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b79614-1783-46df-8ae6-11b9d1f2a38c None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:17:31.317750 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b79614-1783-46df-8ae6-11b9d1f2a38c None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:17:31.318025 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b79614-1783-46df-8ae6-11b9d1f2a38c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:17:31.348135 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466dc9a6-aa4c-4a36-bccd-fcdf5f0933ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:17:31.348805 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466dc9a6-aa4c-4a36-bccd-fcdf5f0933ab None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:17:31.349090 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466dc9a6-aa4c-4a36-bccd-fcdf5f0933ab None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:17:31.349373 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466dc9a6-aa4c-4a36-bccd-fcdf5f0933ab None admin] PATH_INFO: `/v3/roles` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:17:31.350113 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-466dc9a6-aa4c-4a36-bccd-fcdf5f0933ab None admin] RBAC: Adding query filter params (name=admin) {{(pid=80119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:17:31.350420 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-466dc9a6-aa4c-4a36-bccd-fcdf5f0933ab None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:17:31.351866 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-466dc9a6-aa4c-4a36-bccd-fcdf5f0933ab None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:17:31.363541 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 7/15] 10.222.0.49 () {56 vars in 1099 bytes} [Wed Jun 15 10:17:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 10:17:31.369227 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466dc9a6-aa4c-4a36-bccd-fcdf5f0933ab None admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:17:31.384024 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b79614-1783-46df-8ae6-11b9d1f2a38c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:17:31.385650 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b79614-1783-46df-8ae6-11b9d1f2a38c None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:17:31.466185 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466dc9a6-aa4c-4a36-bccd-fcdf5f0933ab None admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:17:31.483860 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 9/16] 10.222.0.49 () {58 vars in 1282 bytes} [Wed Jun 15 10:17:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 15 10:17:31.489286 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b79614-1783-46df-8ae6-11b9d1f2a38c None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:17:31.491911 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:17:31.500345 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75a3bf7-7b2a-4ed8-9c89-11fbcab81b9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:17:31.501227 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75a3bf7-7b2a-4ed8-9c89-11fbcab81b9d None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:17:31.501515 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75a3bf7-7b2a-4ed8-9c89-11fbcab81b9d None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:17:31.501795 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75a3bf7-7b2a-4ed8-9c89-11fbcab81b9d None admin] PATH_INFO: `/v3/users/fcdb2d8e71bd4fe6b9b1431183facba7` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:17:31.503071 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c75a3bf7-7b2a-4ed8-9c89-11fbcab81b9d None admin] RBAC: Authorizing `identity:get_user(user_id=fcdb2d8e71bd4fe6b9b1431183facba7)` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:17:31.504576 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c75a3bf7-7b2a-4ed8-9c89-11fbcab81b9d None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:17:31.506197 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 8/17] 10.222.0.49 () {56 vars in 1177 bytes} [Wed Jun 15 10:17:31 2022] GET /identity/v3/users/fcdb2d8e71bd4fe6b9b1431183facba7 => generated 247 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 10:17:31.511530 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75a3bf7-7b2a-4ed8-9c89-11fbcab81b9d None admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:17:31.526101 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=100 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:17:31.588735 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b79614-1783-46df-8ae6-11b9d1f2a38c None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:17:31.613716 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75a3bf7-7b2a-4ed8-9c89-11fbcab81b9d None admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:17:31.621755 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8cd720-0966-4b14-8511-165ab2499271 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:17:31.622550 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8cd720-0966-4b14-8511-165ab2499271 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:17:31.622842 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8cd720-0966-4b14-8511-165ab2499271 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:17:31.623143 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8cd720-0966-4b14-8511-165ab2499271 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:17:31.644063 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8cd720-0966-4b14-8511-165ab2499271 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:17:31.645537 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8cd720-0966-4b14-8511-165ab2499271 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:17:31.647582 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1dedd2-2542-4270-941a-66c4f229eec2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:17:31.648381 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1dedd2-2542-4270-941a-66c4f229eec2 None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:17:31.648667 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1dedd2-2542-4270-941a-66c4f229eec2 None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:17:31.648945 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1dedd2-2542-4270-941a-66c4f229eec2 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:17:31.650126 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab1dedd2-2542-4270-941a-66c4f229eec2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:17:31.651625 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab1dedd2-2542-4270-941a-66c4f229eec2 None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:17:31.653226 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 9/18] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Jun 15 10:17:31 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 10:17:31.658925 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1dedd2-2542-4270-941a-66c4f229eec2 None admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:17:31.667748 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application [None req-bd8cd720-0966-4b14-8511-165ab2499271 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 10:17:31.667748 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:17:31.667748 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:17:31.667748 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:17:31.667748 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:17:31.667748 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:17:31.667748 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:17:31.667748 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:17:31.667748 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:17:31.667748 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:17:31.667748 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:17:31.667748 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:17:31.667748 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 10:17:31.667748 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 10:17:31.667748 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 10:17:31.668971 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 10:17:31.668971 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:17:31.668971 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:17:31.668971 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:17:31.668971 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:17:31.668971 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:17:31.668971 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:17:31.668971 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:17:31.668971 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application with Lock( Jun 15 10:17:31.668971 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:17:31.668971 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:17:31.668971 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:17:31.668971 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:17:31.668971 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:17:31.668971 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:17:31.668971 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:17:31.668971 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:17:31.669861 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 10:17:31.669861 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 10:17:31.669861 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 10:17:31.669861 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Jun 15 10:17:31.669861 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 10/19] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Jun 15 10:17:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 10:17:31.676123 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8cd720-0966-4b14-8511-165ab2499271 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:17:31.732545 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:17:31.739189 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=100 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:17:31.758287 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1dedd2-2542-4270-941a-66c4f229eec2 None admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:17:31.778044 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8cd720-0966-4b14-8511-165ab2499271 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:17:31.793436 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9769f55d-788c-4f96-8719-d881f74d6d13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:17:31.794106 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9769f55d-788c-4f96-8719-d881f74d6d13 None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:17:31.794393 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9769f55d-788c-4f96-8719-d881f74d6d13 None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:17:31.794679 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9769f55d-788c-4f96-8719-d881f74d6d13 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:17:31.795544 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9769f55d-788c-4f96-8719-d881f74d6d13 None admin] RBAC: Adding query filter params (role.id=8051174e7a7947d5b991ebd793c2e543, scope.domain.id=default, user.id=fcdb2d8e71bd4fe6b9b1431183facba7) {{(pid=80119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:17:31.795911 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9769f55d-788c-4f96-8719-d881f74d6d13 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:17:31.797365 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9769f55d-788c-4f96-8719-d881f74d6d13 None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:17:31.804521 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 10/20] 10.222.0.49 () {56 vars in 1322 bytes} [Wed Jun 15 10:17:31 2022] GET /identity/v3/role_assignments?user.id=fcdb2d8e71bd4fe6b9b1431183facba7&scope.domain.id=default&role.id=8051174e7a7947d5b991ebd793c2e543 => generated 233 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 10:17:31.812811 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ce95c6-2365-4631-82d4-2d765475972d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:17:31.813603 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ce95c6-2365-4631-82d4-2d765475972d None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:17:31.813980 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ce95c6-2365-4631-82d4-2d765475972d None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:17:31.814331 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ce95c6-2365-4631-82d4-2d765475972d None admin] PATH_INFO: `/v3/domains` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:17:31.815196 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ce95c6-2365-4631-82d4-2d765475972d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:17:31.815576 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ce95c6-2365-4631-82d4-2d765475972d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:17:31.817102 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ce95c6-2365-4631-82d4-2d765475972d None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:17:31.837932 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 11/21] 10.222.0.49 () {56 vars in 1109 bytes} [Wed Jun 15 10:17:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 10:17:34.207633 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d4c628-97aa-47b5-a104-6ed196126133 None None] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:17:34.208611 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d4c628-97aa-47b5-a104-6ed196126133 None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:17:34.209184 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d4c628-97aa-47b5-a104-6ed196126133 None None] PATH_INFO: `/` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:17:34.210763 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 11/22] 10.222.0.49 () {54 vars in 926 bytes} [Wed Jun 15 10:17:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 15 10:17:34.216055 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ca183f-40c4-4390-b990-416cace2b953 None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:17:34.216448 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ca183f-40c4-4390-b990-416cace2b953 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:17:34.216612 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ca183f-40c4-4390-b990-416cace2b953 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:17:34.243205 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-77ca183f-40c4-4390-b990-416cace2b953 None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:17:34.244438 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-77ca183f-40c4-4390-b990-416cace2b953 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:17:34.337780 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 12/23] 10.222.0.49 () {58 vars in 1023 bytes} [Wed Jun 15 10:17:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 10:17:34.380775 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d4c628-97aa-47b5-a104-6ed196126133 None None] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:17:34.452555 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67114f0c-4401-49b6-a19e-607f83d59e20 None None] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:17:34.453737 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67114f0c-4401-49b6-a19e-607f83d59e20 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:17:34.454227 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67114f0c-4401-49b6-a19e-607f83d59e20 None None] PATH_INFO: `/` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:17:34.455839 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 13/24] 10.222.0.49 () {54 vars in 926 bytes} [Wed Jun 15 10:17:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:17:34.478483 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5451906-7452-4aa8-96a8-7ffe73cc6c61 None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:17:34.479077 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5451906-7452-4aa8-96a8-7ffe73cc6c61 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:17:34.479600 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5451906-7452-4aa8-96a8-7ffe73cc6c61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:17:34.509764 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-c5451906-7452-4aa8-96a8-7ffe73cc6c61 None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:17:34.511320 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-c5451906-7452-4aa8-96a8-7ffe73cc6c61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:17:34.542178 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d4c628-97aa-47b5-a104-6ed196126133 None None] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:17:34.596269 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:17:34.610679 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70707a2-c213-4f34-b3cb-bf7f7b757b51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:17:34.611876 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70707a2-c213-4f34-b3cb-bf7f7b757b51 None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:17:34.612367 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70707a2-c213-4f34-b3cb-bf7f7b757b51 None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:17:34.612833 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70707a2-c213-4f34-b3cb-bf7f7b757b51 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:17:34.638054 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a70707a2-c213-4f34-b3cb-bf7f7b757b51 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 10:17:34.644689 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 14/25] 10.222.0.49 () {58 vars in 1023 bytes} [Wed Jun 15 10:17:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 15 10:17:34.646103 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf48255-0bdc-433b-b236-b86781bf0558 None None] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:17:34.646400 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf48255-0bdc-433b-b236-b86781bf0558 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:17:34.646683 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf48255-0bdc-433b-b236-b86781bf0558 None None] PATH_INFO: `/` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:17:34.647830 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application [None req-a70707a2-c213-4f34-b3cb-bf7f7b757b51 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 10:17:34.647830 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:17:34.647830 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:17:34.647830 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:17:34.647830 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:17:34.647830 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:17:34.647830 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:17:34.647830 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:17:34.647830 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:17:34.647830 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:17:34.647830 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:17:34.647830 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:17:34.647830 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 10:17:34.647830 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 10:17:34.647830 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 10:17:34.647830 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 10:17:34.648864 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 10:17:34.648864 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 10:17:34.648864 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 10:17:34.648864 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 10:17:34.648864 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:17:34.648864 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:17:34.648864 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:17:34.648864 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:17:34.648864 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:17:34.648864 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:17:34.648864 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:17:34.648864 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application with Lock( Jun 15 10:17:34.648864 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:17:34.648864 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:17:34.648864 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:17:34.648864 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:17:34.649861 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:17:34.649861 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:17:34.649861 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:17:34.649861 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:17:34.649861 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 10:17:34.649861 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 10:17:34.649861 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 10:17:34.649861 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 10:17:34.649861 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 10:17:34.649861 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 10:17:34.649861 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 10:17:34.649861 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application Jun 15 10:17:34.650423 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 15/26] 10.222.0.49 () {54 vars in 926 bytes} [Wed Jun 15 10:17:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:17:34.652512 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 12/27] 10.222.0.49 () {56 vars in 1096 bytes} [Wed Jun 15 10:17:34 2022] GET /identity/v3/roles/admin => generated 83 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:17:34.655316 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e7023a-b646-4a54-95a1-3b76085bb23a None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:17:34.655316 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e7023a-b646-4a54-95a1-3b76085bb23a None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:17:34.655316 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e7023a-b646-4a54-95a1-3b76085bb23a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:17:34.658552 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70707a2-c213-4f34-b3cb-bf7f7b757b51 None admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:17:34.668487 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=100 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:17:34.680395 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-e9e7023a-b646-4a54-95a1-3b76085bb23a None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:17:34.681869 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-e9e7023a-b646-4a54-95a1-3b76085bb23a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:17:34.758068 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 16/28] 10.222.0.49 () {58 vars in 1023 bytes} [Wed Jun 15 10:17:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 10:17:34.759181 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e7023a-b646-4a54-95a1-3b76085bb23a None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:17:34.895597 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70707a2-c213-4f34-b3cb-bf7f7b757b51 None admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:17:34.902516 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e7023a-b646-4a54-95a1-3b76085bb23a None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:17:34.915397 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:17:34.965538 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e949e8-4384-4526-8155-1c8802505202 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:17:34.966236 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e949e8-4384-4526-8155-1c8802505202 None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:17:34.966521 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e949e8-4384-4526-8155-1c8802505202 None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:17:34.966806 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e949e8-4384-4526-8155-1c8802505202 None admin] PATH_INFO: `/v3/roles` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:17:34.967604 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e949e8-4384-4526-8155-1c8802505202 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:17:34.967916 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e949e8-4384-4526-8155-1c8802505202 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:17:34.969419 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e949e8-4384-4526-8155-1c8802505202 None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:17:34.973524 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb47e4dd-c317-4440-90a3-1e2a6c88b30c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:17:34.974228 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb47e4dd-c317-4440-90a3-1e2a6c88b30c None admin] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:17:34.974514 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb47e4dd-c317-4440-90a3-1e2a6c88b30c None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:17:34.974791 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb47e4dd-c317-4440-90a3-1e2a6c88b30c None admin] PATH_INFO: `/v3/roles` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:17:34.975594 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb47e4dd-c317-4440-90a3-1e2a6c88b30c None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:17:34.977037 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb47e4dd-c317-4440-90a3-1e2a6c88b30c None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:17:34.991617 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 13/29] 10.222.0.49 () {56 vars in 1099 bytes} [Wed Jun 15 10:17:34 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 333 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:17:34.992823 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e949e8-4384-4526-8155-1c8802505202 None admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:17:35.088426 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=100 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:17:35.125636 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e949e8-4384-4526-8155-1c8802505202 None admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:17:35.134221 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 17/30] 10.222.0.49 () {60 vars in 1131 bytes} [Wed Jun 15 10:17:34 2022] POST /identity/v3/roles => generated 219 bytes in 375 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 10:17:35.135256 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb47e4dd-c317-4440-90a3-1e2a6c88b30c None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:17:35.224608 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea0a99f-edd1-4218-b8c4-845f92c56460 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:17:35.225448 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea0a99f-edd1-4218-b8c4-845f92c56460 None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:17:35.225731 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea0a99f-edd1-4218-b8c4-845f92c56460 None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:17:35.225997 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea0a99f-edd1-4218-b8c4-845f92c56460 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:17:35.249379 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb47e4dd-c317-4440-90a3-1e2a6c88b30c None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:17:35.256878 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea0a99f-edd1-4218-b8c4-845f92c56460 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:17:35.258417 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea0a99f-edd1-4218-b8c4-845f92c56460 None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:17:35.289308 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a385c762-6fbc-447d-a061-df810b15d277 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:17:35.290443 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a385c762-6fbc-447d-a061-df810b15d277 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:17:35.290891 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a385c762-6fbc-447d-a061-df810b15d277 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:17:35.291339 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a385c762-6fbc-447d-a061-df810b15d277 None admin] PATH_INFO: `/v3/users/fcdb2d8e71bd4fe6b9b1431183facba7` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:17:35.293393 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a385c762-6fbc-447d-a061-df810b15d277 None admin] RBAC: Authorizing `identity:get_user(user_id=fcdb2d8e71bd4fe6b9b1431183facba7)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:17:35.295149 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a385c762-6fbc-447d-a061-df810b15d277 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:17:35.298412 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 18/31] 10.222.0.49 () {56 vars in 1177 bytes} [Wed Jun 15 10:17:35 2022] GET /identity/v3/users/fcdb2d8e71bd4fe6b9b1431183facba7 => generated 247 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:17:35.299943 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ee2a45-69fc-41df-9139-d65a035d6d46 None None] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:17:35.300373 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ee2a45-69fc-41df-9139-d65a035d6d46 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:17:35.300812 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ee2a45-69fc-41df-9139-d65a035d6d46 None None] PATH_INFO: `/` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:17:35.300921 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application [None req-4ea0a99f-edd1-4218-b8c4-845f92c56460 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 10:17:35.300921 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:17:35.300921 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:17:35.300921 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:17:35.300921 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:17:35.300921 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:17:35.300921 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:17:35.300921 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:17:35.300921 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:17:35.300921 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:17:35.300921 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:17:35.300921 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:17:35.300921 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 10:17:35.300921 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 10:17:35.300921 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 10:17:35.301675 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 10:17:35.301675 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:17:35.301675 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:17:35.301675 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:17:35.301675 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:17:35.301675 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:17:35.301675 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:17:35.301675 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:17:35.301675 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application with Lock( Jun 15 10:17:35.301675 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:17:35.301675 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:17:35.301675 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:17:35.301675 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:17:35.301675 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:17:35.301675 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:17:35.301675 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:17:35.301675 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:17:35.302483 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 10:17:35.302483 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 10:17:35.302483 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 10:17:35.302483 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application Jun 15 10:17:35.302483 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 14/32] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Jun 15 10:17:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 310 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:17:35.304527 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 19/33] 10.222.0.49 () {54 vars in 926 bytes} [Wed Jun 15 10:17:35 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:17:35.305566 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea0a99f-edd1-4218-b8c4-845f92c56460 None admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:17:35.314497 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9722782-9c4d-46d1-82e3-f0e73fa0d5a8 None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:17:35.315016 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9722782-9c4d-46d1-82e3-f0e73fa0d5a8 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:17:35.315503 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9722782-9c4d-46d1-82e3-f0e73fa0d5a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:17:35.337657 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-f9722782-9c4d-46d1-82e3-f0e73fa0d5a8 None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:17:35.339448 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-f9722782-9c4d-46d1-82e3-f0e73fa0d5a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:17:35.372714 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:17:35.426505 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 20/34] 10.222.0.49 () {58 vars in 1023 bytes} [Wed Jun 15 10:17:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 10:17:35.431916 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9722782-9c4d-46d1-82e3-f0e73fa0d5a8 None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:17:35.455370 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea0a99f-edd1-4218-b8c4-845f92c56460 None admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:17:35.477657 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=100 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:17:35.503589 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f272131-bd61-48a7-8aa9-0822b6e36d95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:17:35.504793 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f272131-bd61-48a7-8aa9-0822b6e36d95 None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:17:35.505256 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f272131-bd61-48a7-8aa9-0822b6e36d95 None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:17:35.505708 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f272131-bd61-48a7-8aa9-0822b6e36d95 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:17:35.507473 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f272131-bd61-48a7-8aa9-0822b6e36d95 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:17:35.509258 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f272131-bd61-48a7-8aa9-0822b6e36d95 None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:17:35.512242 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 15/35] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Jun 15 10:17:35 2022] GET /identity/v3/domains/default => generated 199 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:17:35.513480 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f272131-bd61-48a7-8aa9-0822b6e36d95 None admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:17:35.553120 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9722782-9c4d-46d1-82e3-f0e73fa0d5a8 None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:17:35.591959 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eead93df-92ad-4408-8904-7de3a96bd9e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:17:35.592558 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eead93df-92ad-4408-8904-7de3a96bd9e3 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:17:35.592842 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eead93df-92ad-4408-8904-7de3a96bd9e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:17:35.593124 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eead93df-92ad-4408-8904-7de3a96bd9e3 None admin] PATH_INFO: `/v3/domains` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:17:35.593892 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eead93df-92ad-4408-8904-7de3a96bd9e3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:17:35.594220 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eead93df-92ad-4408-8904-7de3a96bd9e3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:17:35.595634 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eead93df-92ad-4408-8904-7de3a96bd9e3 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:17:35.607548 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 21/36] 10.222.0.49 () {56 vars in 1109 bytes} [Wed Jun 15 10:17:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:17:35.612641 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eead93df-92ad-4408-8904-7de3a96bd9e3 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:17:35.640074 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:17:35.650575 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f272131-bd61-48a7-8aa9-0822b6e36d95 None admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:17:35.688258 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9306fc-e1e7-4fb5-8103-715a1cbb9f92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:17:35.688938 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9306fc-e1e7-4fb5-8103-715a1cbb9f92 None admin] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:17:35.689224 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9306fc-e1e7-4fb5-8103-715a1cbb9f92 None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:17:35.689497 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9306fc-e1e7-4fb5-8103-715a1cbb9f92 None admin] PATH_INFO: `/v3/roles` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:17:35.690234 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc9306fc-e1e7-4fb5-8103-715a1cbb9f92 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:17:35.691908 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc9306fc-e1e7-4fb5-8103-715a1cbb9f92 None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:17:35.730188 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eead93df-92ad-4408-8904-7de3a96bd9e3 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:17:35.770033 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844771b2-f37e-4a59-81ff-46c8fb12a9bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:17:35.771680 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844771b2-f37e-4a59-81ff-46c8fb12a9bf None admin] REQUEST_METHOD: `PUT` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:17:35.772305 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844771b2-f37e-4a59-81ff-46c8fb12a9bf None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:17:35.772797 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844771b2-f37e-4a59-81ff-46c8fb12a9bf None admin] PATH_INFO: `/v3/domains/default/users/fcdb2d8e71bd4fe6b9b1431183facba7/roles/8051174e7a7947d5b991ebd793c2e543` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:17:35.775398 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-844771b2-f37e-4a59-81ff-46c8fb12a9bf None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=fcdb2d8e71bd4fe6b9b1431183facba7, role_id=8051174e7a7947d5b991ebd793c2e543)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:17:35.777454 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-844771b2-f37e-4a59-81ff-46c8fb12a9bf None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:17:35.801456 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 22/37] 10.222.0.49 () {58 vars in 1361 bytes} [Wed Jun 15 10:17:35 2022] PUT /identity/v3/domains/default/users/fcdb2d8e71bd4fe6b9b1431183facba7/roles/8051174e7a7947d5b991ebd793c2e543 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:17:35.802852 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844771b2-f37e-4a59-81ff-46c8fb12a9bf None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:17:35.946786 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 16/38] 10.222.0.49 () {60 vars in 1131 bytes} [Wed Jun 15 10:17:35 2022] POST /identity/v3/roles => generated 225 bytes in 434 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 10:17:35.948470 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c570c0-b78e-4ec4-9727-dbf2acdc120d None None] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:17:35.948782 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c570c0-b78e-4ec4-9727-dbf2acdc120d None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:17:35.949041 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c570c0-b78e-4ec4-9727-dbf2acdc120d None None] PATH_INFO: `/` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:17:35.950850 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 17/39] 10.222.0.49 () {54 vars in 926 bytes} [Wed Jun 15 10:17:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:17:35.956539 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc79249-22e9-4db6-9eb5-cf8c6640cd5f None None] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:17:35.956836 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc79249-22e9-4db6-9eb5-cf8c6640cd5f None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:17:35.957129 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc79249-22e9-4db6-9eb5-cf8c6640cd5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:17:36.000996 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.auth.core [None req-edc79249-22e9-4db6-9eb5-cf8c6640cd5f None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:17:36.002720 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-edc79249-22e9-4db6-9eb5-cf8c6640cd5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:17:36.047320 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844771b2-f37e-4a59-81ff-46c8fb12a9bf None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:17:36.050113 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:17:36.059854 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:17:36.095896 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10223c36-ba27-49d2-9e68-399f2dde04d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:17:36.096598 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10223c36-ba27-49d2-9e68-399f2dde04d8 None admin] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:17:36.096890 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10223c36-ba27-49d2-9e68-399f2dde04d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:17:36.097173 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10223c36-ba27-49d2-9e68-399f2dde04d8 None admin] PATH_INFO: `/v3/projects` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:17:36.097864 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 18/40] 10.222.0.49 () {58 vars in 1023 bytes} [Wed Jun 15 10:17:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 10:17:36.097971 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10223c36-ba27-49d2-9e68-399f2dde04d8 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:17:36.099206 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474d2e9b-a935-4ac1-b847-8923082e2661 None None] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:17:36.099406 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10223c36-ba27-49d2-9e68-399f2dde04d8 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:17:36.099572 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474d2e9b-a935-4ac1-b847-8923082e2661 None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:17:36.099827 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474d2e9b-a935-4ac1-b847-8923082e2661 None None] PATH_INFO: `/` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:17:36.101301 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 19/41] 10.222.0.49 () {54 vars in 926 bytes} [Wed Jun 15 10:17:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:17:36.107398 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e597e80-2db6-4d87-9d8b-d3240e17454b None None] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:17:36.107689 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e597e80-2db6-4d87-9d8b-d3240e17454b None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:17:36.107973 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e597e80-2db6-4d87-9d8b-d3240e17454b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:17:36.137317 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.auth.core [None req-5e597e80-2db6-4d87-9d8b-d3240e17454b None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:17:36.138552 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-5e597e80-2db6-4d87-9d8b-d3240e17454b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:17:36.139849 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 23/42] 10.222.0.49 () {60 vars in 1141 bytes} [Wed Jun 15 10:17:35 2022] POST /identity/v3/projects => generated 301 bytes in 337 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 10:17:36.140630 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10223c36-ba27-49d2-9e68-399f2dde04d8 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:17:36.243010 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 20/43] 10.222.0.49 () {58 vars in 1023 bytes} [Wed Jun 15 10:17:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 15 10:17:36.262730 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e597e80-2db6-4d87-9d8b-d3240e17454b None None] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:17:36.278368 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10223c36-ba27-49d2-9e68-399f2dde04d8 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:17:36.319556 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb2bd24-45ea-4ddc-b2b7-4bfcf933b4e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:17:36.320561 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb2bd24-45ea-4ddc-b2b7-4bfcf933b4e5 None admin] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:17:36.320857 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb2bd24-45ea-4ddc-b2b7-4bfcf933b4e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:17:36.321160 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb2bd24-45ea-4ddc-b2b7-4bfcf933b4e5 None admin] PATH_INFO: `/v3/roles` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:17:36.322319 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb2bd24-45ea-4ddc-b2b7-4bfcf933b4e5 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:17:36.323793 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb2bd24-45ea-4ddc-b2b7-4bfcf933b4e5 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:17:36.328108 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:17:36.346607 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 24/44] 10.222.0.49 () {60 vars in 1131 bytes} [Wed Jun 15 10:17:36 2022] POST /identity/v3/roles => generated 223 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 10:17:36.386178 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e597e80-2db6-4d87-9d8b-d3240e17454b None None] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:17:36.396043 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=100 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:17:36.433585 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c0380d-3cc8-4070-952e-b7b2a7424f17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:17:36.434709 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c0380d-3cc8-4070-952e-b7b2a7424f17 None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:17:36.435180 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c0380d-3cc8-4070-952e-b7b2a7424f17 None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:17:36.435611 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c0380d-3cc8-4070-952e-b7b2a7424f17 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:17:36.437124 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c0380d-3cc8-4070-952e-b7b2a7424f17 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:17:36.438929 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c0380d-3cc8-4070-952e-b7b2a7424f17 None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:17:36.441175 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 21/45] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Jun 15 10:17:36 2022] GET /identity/v3/domains/default => generated 199 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:17:36.446845 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb2bd24-45ea-4ddc-b2b7-4bfcf933b4e5 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:17:36.570579 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb2bd24-45ea-4ddc-b2b7-4bfcf933b4e5 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:17:36.605645 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975ed9c1-b0ed-499a-8f0d-b669d2ab52aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:17:36.606315 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975ed9c1-b0ed-499a-8f0d-b669d2ab52aa None admin] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:17:36.606600 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975ed9c1-b0ed-499a-8f0d-b669d2ab52aa None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:17:36.606876 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975ed9c1-b0ed-499a-8f0d-b669d2ab52aa None admin] PATH_INFO: `/v3/projects` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:17:36.607745 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975ed9c1-b0ed-499a-8f0d-b669d2ab52aa None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:17:36.609245 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975ed9c1-b0ed-499a-8f0d-b669d2ab52aa None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:17:36.634887 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 25/46] 10.222.0.49 () {60 vars in 1141 bytes} [Wed Jun 15 10:17:36 2022] POST /identity/v3/projects => generated 312 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 10:17:38.093942 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e0faa4-43fa-4ffb-9f88-e67773ef55e8 None None] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:17:38.094429 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e0faa4-43fa-4ffb-9f88-e67773ef55e8 None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:17:38.094429 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e0faa4-43fa-4ffb-9f88-e67773ef55e8 None None] PATH_INFO: `/` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:17:38.095849 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 22/47] 10.222.0.49 () {54 vars in 926 bytes} [Wed Jun 15 10:17:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:17:38.101200 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bb782e-73af-459a-951b-3ec534be0dfa None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:17:38.101611 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bb782e-73af-459a-951b-3ec534be0dfa None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:17:38.101765 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bb782e-73af-459a-951b-3ec534be0dfa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:17:38.118070 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-06bb782e-73af-459a-951b-3ec534be0dfa None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:17:38.119296 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-06bb782e-73af-459a-951b-3ec534be0dfa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:17:38.162945 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:17:38.186200 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 26/48] 10.222.0.49 () {58 vars in 1023 bytes} [Wed Jun 15 10:17:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 10:17:38.209670 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e0faa4-43fa-4ffb-9f88-e67773ef55e8 None None] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:17:38.309482 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e0faa4-43fa-4ffb-9f88-e67773ef55e8 None None] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:17:38.342954 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b41f3eb-28b4-49db-bd58-faf755d7a124 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:17:38.343802 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b41f3eb-28b4-49db-bd58-faf755d7a124 None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:17:38.344141 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b41f3eb-28b4-49db-bd58-faf755d7a124 None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:17:38.344394 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b41f3eb-28b4-49db-bd58-faf755d7a124 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:17:38.354437 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b41f3eb-28b4-49db-bd58-faf755d7a124 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 10:17:38.362070 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application [None req-2b41f3eb-28b4-49db-bd58-faf755d7a124 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 10:17:38.362070 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:17:38.362070 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:17:38.362070 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:17:38.362070 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:17:38.362070 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:17:38.362070 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:17:38.362070 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:17:38.362070 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:17:38.362070 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:17:38.362070 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:17:38.362070 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:17:38.362070 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 10:17:38.362070 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 10:17:38.362070 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 10:17:38.362070 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 10:17:38.362905 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 10:17:38.362905 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 10:17:38.362905 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 10:17:38.362905 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 10:17:38.362905 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:17:38.362905 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:17:38.362905 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:17:38.362905 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:17:38.362905 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:17:38.362905 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:17:38.362905 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:17:38.362905 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application with Lock( Jun 15 10:17:38.362905 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:17:38.362905 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:17:38.362905 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:17:38.362905 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:17:38.363712 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:17:38.363712 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:17:38.363712 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:17:38.363712 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:17:38.363712 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 10:17:38.363712 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 10:17:38.363712 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 10:17:38.363712 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 10:17:38.363712 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 10:17:38.363712 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 10:17:38.363712 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 10:17:38.363712 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application Jun 15 10:17:38.363712 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 23/49] 10.222.0.49 () {56 vars in 1096 bytes} [Wed Jun 15 10:17:38 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:17:38.369979 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06bb782e-73af-459a-951b-3ec534be0dfa None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:17:38.467465 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06bb782e-73af-459a-951b-3ec534be0dfa None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:17:38.501221 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3f9bc8-7ba7-4f2d-b2c7-16bb9976c4ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:17:38.501893 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3f9bc8-7ba7-4f2d-b2c7-16bb9976c4ad None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:17:38.502302 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3f9bc8-7ba7-4f2d-b2c7-16bb9976c4ad None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:17:38.502762 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3f9bc8-7ba7-4f2d-b2c7-16bb9976c4ad None admin] PATH_INFO: `/v3/roles` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:17:38.503816 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e3f9bc8-7ba7-4f2d-b2c7-16bb9976c4ad None admin] RBAC: Adding query filter params (name=admin) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:17:38.504319 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e3f9bc8-7ba7-4f2d-b2c7-16bb9976c4ad None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:17:38.505930 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e3f9bc8-7ba7-4f2d-b2c7-16bb9976c4ad None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:17:38.518460 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 27/50] 10.222.0.49 () {56 vars in 1099 bytes} [Wed Jun 15 10:17:38 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:17:38.523767 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b41f3eb-28b4-49db-bd58-faf755d7a124 None admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:17:38.557524 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=100 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:17:38.623027 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b41f3eb-28b4-49db-bd58-faf755d7a124 None admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:17:38.655813 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f76d954-04c0-4a0a-90d7-2647674f09f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:17:38.656685 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f76d954-04c0-4a0a-90d7-2647674f09f3 None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:17:38.656973 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f76d954-04c0-4a0a-90d7-2647674f09f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:17:38.657249 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f76d954-04c0-4a0a-90d7-2647674f09f3 None admin] PATH_INFO: `/v3/users/fcdb2d8e71bd4fe6b9b1431183facba7` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:17:38.658450 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f76d954-04c0-4a0a-90d7-2647674f09f3 None admin] RBAC: Authorizing `identity:get_user(user_id=fcdb2d8e71bd4fe6b9b1431183facba7)` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:17:38.659966 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f76d954-04c0-4a0a-90d7-2647674f09f3 None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:17:38.661668 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 24/51] 10.222.0.49 () {56 vars in 1177 bytes} [Wed Jun 15 10:17:38 2022] GET /identity/v3/users/fcdb2d8e71bd4fe6b9b1431183facba7 => generated 247 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:17:38.667368 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3f9bc8-7ba7-4f2d-b2c7-16bb9976c4ad None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:17:38.712357 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:17:38.734617 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cffc48-10be-435d-b7ec-6ec0357f6dbb None None] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:17:38.734910 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cffc48-10be-435d-b7ec-6ec0357f6dbb None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:17:38.735221 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cffc48-10be-435d-b7ec-6ec0357f6dbb None None] PATH_INFO: `/` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:17:38.736594 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 25/52] 10.222.0.49 () {54 vars in 926 bytes} [Wed Jun 15 10:17:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:17:38.741307 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae78aece-2ff2-49db-a934-4e844bec3578 None None] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:17:38.741603 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae78aece-2ff2-49db-a934-4e844bec3578 None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:17:38.741890 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae78aece-2ff2-49db-a934-4e844bec3578 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:17:38.758688 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.auth.core [None req-ae78aece-2ff2-49db-a934-4e844bec3578 None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:17:38.759915 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-ae78aece-2ff2-49db-a934-4e844bec3578 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:17:38.770868 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3f9bc8-7ba7-4f2d-b2c7-16bb9976c4ad None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:17:38.805431 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba806920-ae73-4e4b-9cc0-8f34f7cb5511 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:17:38.806243 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba806920-ae73-4e4b-9cc0-8f34f7cb5511 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:17:38.806532 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba806920-ae73-4e4b-9cc0-8f34f7cb5511 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:17:38.806824 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba806920-ae73-4e4b-9cc0-8f34f7cb5511 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:17:38.808041 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba806920-ae73-4e4b-9cc0-8f34f7cb5511 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:17:38.809493 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba806920-ae73-4e4b-9cc0-8f34f7cb5511 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:17:38.811141 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 28/53] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Jun 15 10:17:38 2022] GET /identity/v3/domains/default => generated 199 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:17:38.817283 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba806920-ae73-4e4b-9cc0-8f34f7cb5511 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:17:38.829864 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 26/54] 10.222.0.49 () {58 vars in 1023 bytes} [Wed Jun 15 10:17:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 15 10:17:38.851894 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae78aece-2ff2-49db-a934-4e844bec3578 None None] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:17:38.881688 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=100 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:17:38.915403 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba806920-ae73-4e4b-9cc0-8f34f7cb5511 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:17:38.924413 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:17:38.950073 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae78aece-2ff2-49db-a934-4e844bec3578 None None] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:17:38.950317 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04910bbf-ec31-4306-a8a5-7cc9ead7309a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:17:38.950979 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04910bbf-ec31-4306-a8a5-7cc9ead7309a None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:17:38.951309 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04910bbf-ec31-4306-a8a5-7cc9ead7309a None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:17:38.951589 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04910bbf-ec31-4306-a8a5-7cc9ead7309a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:17:38.952452 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04910bbf-ec31-4306-a8a5-7cc9ead7309a None admin] RBAC: Adding query filter params (role.id=8051174e7a7947d5b991ebd793c2e543, scope.domain.id=default, user.id=fcdb2d8e71bd4fe6b9b1431183facba7) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:17:38.952767 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04910bbf-ec31-4306-a8a5-7cc9ead7309a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:17:38.954217 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04910bbf-ec31-4306-a8a5-7cc9ead7309a None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:17:38.962218 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 29/55] 10.222.0.49 () {56 vars in 1322 bytes} [Wed Jun 15 10:17:38 2022] GET /identity/v3/role_assignments?user.id=fcdb2d8e71bd4fe6b9b1431183facba7&scope.domain.id=default&role.id=8051174e7a7947d5b991ebd793c2e543 => generated 530 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 10:17:38.984991 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642cd98e-3251-484d-968f-c6389edbe240 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:17:38.985828 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642cd98e-3251-484d-968f-c6389edbe240 None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:17:38.986121 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642cd98e-3251-484d-968f-c6389edbe240 None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:17:38.986402 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642cd98e-3251-484d-968f-c6389edbe240 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:17:38.987713 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642cd98e-3251-484d-968f-c6389edbe240 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:17:38.989245 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642cd98e-3251-484d-968f-c6389edbe240 None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:17:38.990985 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 27/56] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Jun 15 10:17:38 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 10:17:38.997029 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04910bbf-ec31-4306-a8a5-7cc9ead7309a None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:17:39.102197 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04910bbf-ec31-4306-a8a5-7cc9ead7309a None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:17:39.138485 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6336800-8ac3-43f7-8b72-638953bee991 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:17:39.139079 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6336800-8ac3-43f7-8b72-638953bee991 None admin] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:17:39.139398 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6336800-8ac3-43f7-8b72-638953bee991 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:17:39.139677 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6336800-8ac3-43f7-8b72-638953bee991 None admin] PATH_INFO: `/v3/projects` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:17:39.140390 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6336800-8ac3-43f7-8b72-638953bee991 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:17:39.141843 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6336800-8ac3-43f7-8b72-638953bee991 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:17:39.169163 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 30/57] 10.222.0.49 () {60 vars in 1140 bytes} [Wed Jun 15 10:17:38 2022] POST /identity/v3/projects => generated 298 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 10:17:41.248101 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba1be70-2036-4025-aa77-0e89c342ec40 None None] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:17:41.248584 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba1be70-2036-4025-aa77-0e89c342ec40 None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:17:41.248584 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba1be70-2036-4025-aa77-0e89c342ec40 None None] PATH_INFO: `/` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:17:41.249937 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 28/58] 10.222.0.49 () {54 vars in 926 bytes} [Wed Jun 15 10:17:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:17:41.255037 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b40502-fab4-4c11-a983-07f4674b6d0c None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:17:41.255464 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b40502-fab4-4c11-a983-07f4674b6d0c None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:17:41.255641 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b40502-fab4-4c11-a983-07f4674b6d0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:17:41.272383 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-a6b40502-fab4-4c11-a983-07f4674b6d0c None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:17:41.273602 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-a6b40502-fab4-4c11-a983-07f4674b6d0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:17:41.290995 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:17:41.340096 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 31/59] 10.222.0.49 () {58 vars in 1023 bytes} [Wed Jun 15 10:17:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 10:17:41.363516 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba1be70-2036-4025-aa77-0e89c342ec40 None None] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:17:41.458930 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=100 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:17:41.463117 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba1be70-2036-4025-aa77-0e89c342ec40 None None] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:17:41.494956 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9a8b43-dfd1-4dc8-a22d-1ef2b23a341c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:17:41.495774 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9a8b43-dfd1-4dc8-a22d-1ef2b23a341c None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:17:41.496067 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9a8b43-dfd1-4dc8-a22d-1ef2b23a341c None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:17:41.496351 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9a8b43-dfd1-4dc8-a22d-1ef2b23a341c None admin] PATH_INFO: `/v3/domains/default` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:17:41.497526 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab9a8b43-dfd1-4dc8-a22d-1ef2b23a341c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:17:41.499018 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab9a8b43-dfd1-4dc8-a22d-1ef2b23a341c None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:17:41.501120 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 29/60] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Jun 15 10:17:41 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:17:41.507193 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b40502-fab4-4c11-a983-07f4674b6d0c None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:17:41.604651 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b40502-fab4-4c11-a983-07f4674b6d0c None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:17:41.637313 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8667bc61-73e6-4cf5-9812-6b2760a16904 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:17:41.637986 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8667bc61-73e6-4cf5-9812-6b2760a16904 None admin] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:17:41.638273 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8667bc61-73e6-4cf5-9812-6b2760a16904 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:17:41.638555 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8667bc61-73e6-4cf5-9812-6b2760a16904 None admin] PATH_INFO: `/v3/users` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:17:41.639288 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8667bc61-73e6-4cf5-9812-6b2760a16904 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:17:41.641319 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8667bc61-73e6-4cf5-9812-6b2760a16904 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:17:41.665763 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 32/61] 10.222.0.49 () {60 vars in 1132 bytes} [Wed Jun 15 10:17:41 2022] POST /identity/v3/users => generated 275 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 10:17:44.994448 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4a7130-4a6d-4cc8-9fe9-37ee7710890e None None] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:17:44.995327 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4a7130-4a6d-4cc8-9fe9-37ee7710890e None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:17:44.995769 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4a7130-4a6d-4cc8-9fe9-37ee7710890e None None] PATH_INFO: `/` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:17:44.997543 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 30/62] 10.222.0.49 () {54 vars in 926 bytes} [Wed Jun 15 10:17:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:17:45.003199 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896deb26-01fa-410e-8ce2-ac6c951dd30e None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:17:45.004048 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896deb26-01fa-410e-8ce2-ac6c951dd30e None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:17:45.004515 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896deb26-01fa-410e-8ce2-ac6c951dd30e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:17:45.028253 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-896deb26-01fa-410e-8ce2-ac6c951dd30e None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:17:45.029712 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-896deb26-01fa-410e-8ce2-ac6c951dd30e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:17:45.033731 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:17:45.121629 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 33/63] 10.222.0.49 () {58 vars in 1023 bytes} [Wed Jun 15 10:17:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 10:17:45.136985 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6bcc1f-5aa7-4757-9fd5-6367cfddf1f2 None None] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:17:45.137277 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6bcc1f-5aa7-4757-9fd5-6367cfddf1f2 None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:17:45.137562 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6bcc1f-5aa7-4757-9fd5-6367cfddf1f2 None None] PATH_INFO: `/` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:17:45.138939 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 31/64] 10.222.0.49 () {54 vars in 926 bytes} [Wed Jun 15 10:17:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:17:45.144903 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58426b0f-01b5-486e-a1d3-3ff9ea3dd6cd None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:17:45.145199 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58426b0f-01b5-486e-a1d3-3ff9ea3dd6cd None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:17:45.145489 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58426b0f-01b5-486e-a1d3-3ff9ea3dd6cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:17:45.155663 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f896907-3ea2-4d1a-88c4-8dec67640465 None None] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:17:45.155954 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f896907-3ea2-4d1a-88c4-8dec67640465 None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:17:45.156238 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f896907-3ea2-4d1a-88c4-8dec67640465 None None] PATH_INFO: `/` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:17:45.157599 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 32/65] 10.222.0.49 () {54 vars in 926 bytes} [Wed Jun 15 10:17:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 15 10:17:45.163518 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-58426b0f-01b5-486e-a1d3-3ff9ea3dd6cd None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:17:45.163724 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f896907-3ea2-4d1a-88c4-8dec67640465 None None] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:17:45.164719 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-58426b0f-01b5-486e-a1d3-3ff9ea3dd6cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:17:45.234977 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 34/66] 10.222.0.49 () {58 vars in 1023 bytes} [Wed Jun 15 10:17:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 10:17:45.243107 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6a2473-79ef-44b8-8a89-2d966891e85d None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:17:45.243587 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6a2473-79ef-44b8-8a89-2d966891e85d None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:17:45.244017 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6a2473-79ef-44b8-8a89-2d966891e85d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:17:45.270438 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-8d6a2473-79ef-44b8-8a89-2d966891e85d None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:17:45.272021 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-8d6a2473-79ef-44b8-8a89-2d966891e85d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:17:45.276779 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f896907-3ea2-4d1a-88c4-8dec67640465 None None] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:17:45.320458 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74039fa-0e11-4cad-b2a4-68992a1f4aa1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:17:45.321549 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74039fa-0e11-4cad-b2a4-68992a1f4aa1 None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:17:45.322006 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74039fa-0e11-4cad-b2a4-68992a1f4aa1 None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:17:45.322523 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74039fa-0e11-4cad-b2a4-68992a1f4aa1 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:17:45.334449 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b74039fa-0e11-4cad-b2a4-68992a1f4aa1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 10:17:45.342470 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application [None req-b74039fa-0e11-4cad-b2a4-68992a1f4aa1 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 15 10:17:45.342470 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:17:45.342470 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:17:45.342470 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:17:45.342470 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:17:45.342470 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:17:45.342470 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:17:45.342470 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:17:45.342470 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:17:45.342470 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:17:45.342470 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:17:45.342470 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:17:45.342470 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 10:17:45.342470 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 10:17:45.342470 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 10:17:45.342470 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 10:17:45.343592 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 10:17:45.343592 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 10:17:45.343592 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 10:17:45.343592 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 10:17:45.343592 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:17:45.343592 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:17:45.343592 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:17:45.343592 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:17:45.343592 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:17:45.343592 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:17:45.343592 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:17:45.343592 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application with Lock( Jun 15 10:17:45.343592 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:17:45.343592 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:17:45.343592 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:17:45.343592 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:17:45.344480 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:17:45.344480 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:17:45.344480 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:17:45.344480 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:17:45.344480 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 10:17:45.344480 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 10:17:45.344480 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 10:17:45.344480 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 10:17:45.344480 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 10:17:45.344480 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 10:17:45.344480 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 15 10:17:45.344480 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application Jun 15 10:17:45.345393 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:17:45.346624 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 33/67] 10.222.0.49 () {56 vars in 1099 bytes} [Wed Jun 15 10:17:45 2022] GET /identity/v3/roles/member => generated 84 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 10:17:45.347634 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74039fa-0e11-4cad-b2a4-68992a1f4aa1 None admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:17:45.354779 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 35/68] 10.222.0.49 () {58 vars in 1023 bytes} [Wed Jun 15 10:17:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 10:17:45.356261 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32258e4b-4c62-47a2-9940-23968d1d089f None None] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:17:45.356561 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32258e4b-4c62-47a2-9940-23968d1d089f None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:17:45.356847 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32258e4b-4c62-47a2-9940-23968d1d089f None None] PATH_INFO: `/` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:17:45.358126 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 36/69] 10.222.0.49 () {54 vars in 926 bytes} [Wed Jun 15 10:17:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:17:45.358866 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32258e4b-4c62-47a2-9940-23968d1d089f None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:17:45.378666 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=100 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:17:45.456186 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74039fa-0e11-4cad-b2a4-68992a1f4aa1 None admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:17:45.472959 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32258e4b-4c62-47a2-9940-23968d1d089f None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:17:45.491897 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de430bc-e4cc-4888-ba64-2e62362de527 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:17:45.493168 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de430bc-e4cc-4888-ba64-2e62362de527 None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:17:45.493461 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de430bc-e4cc-4888-ba64-2e62362de527 None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:17:45.493742 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de430bc-e4cc-4888-ba64-2e62362de527 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:17:45.508451 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de430bc-e4cc-4888-ba64-2e62362de527 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 10:17:45.511135 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc9361b-28dc-4491-b7fd-42091b900156 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:17:45.511859 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc9361b-28dc-4491-b7fd-42091b900156 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:17:45.512143 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc9361b-28dc-4491-b7fd-42091b900156 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:17:45.512418 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc9361b-28dc-4491-b7fd-42091b900156 None admin] PATH_INFO: `/v3/roles` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:17:45.513191 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc9361b-28dc-4491-b7fd-42091b900156 None admin] RBAC: Adding query filter params (name=member) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:17:45.513502 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc9361b-28dc-4491-b7fd-42091b900156 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:17:45.514979 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc9361b-28dc-4491-b7fd-42091b900156 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:17:45.516359 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application [None req-2de430bc-e4cc-4888-ba64-2e62362de527 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 10:17:45.516359 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:17:45.516359 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:17:45.516359 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:17:45.516359 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:17:45.516359 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:17:45.516359 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:17:45.516359 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:17:45.516359 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:17:45.516359 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:17:45.516359 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:17:45.516359 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:17:45.516359 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 10:17:45.516359 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 10:17:45.516359 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 10:17:45.516359 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 10:17:45.517194 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 10:17:45.517194 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 10:17:45.517194 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 10:17:45.517194 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 10:17:45.517194 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:17:45.517194 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:17:45.517194 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:17:45.517194 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:17:45.517194 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:17:45.517194 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:17:45.517194 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:17:45.517194 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application with Lock( Jun 15 10:17:45.517194 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:17:45.517194 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:17:45.517194 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:17:45.517194 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:17:45.517958 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:17:45.517958 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:17:45.517958 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:17:45.517958 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:17:45.517958 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 10:17:45.517958 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 10:17:45.517958 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 10:17:45.517958 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 10:17:45.517958 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 10:17:45.517958 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 10:17:45.517958 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 10:17:45.517958 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application Jun 15 10:17:45.517958 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 34/70] 10.222.0.49 () {56 vars in 1096 bytes} [Wed Jun 15 10:17:45 2022] GET /identity/v3/roles/admin => generated 83 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:17:45.518935 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d58d41-cfe8-430f-9755-e08bc5751729 None None] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:17:45.519397 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d58d41-cfe8-430f-9755-e08bc5751729 None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:17:45.519883 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d58d41-cfe8-430f-9755-e08bc5751729 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:17:45.530127 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 37/71] 10.222.0.49 () {56 vars in 1101 bytes} [Wed Jun 15 10:17:45 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:17:45.530975 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc9361b-28dc-4491-b7fd-42091b900156 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:17:45.540028 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.auth.core [None req-47d58d41-cfe8-430f-9755-e08bc5751729 None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:17:45.541245 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-47d58d41-cfe8-430f-9755-e08bc5751729 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:17:45.594994 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:17:45.609015 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 35/72] 10.222.0.49 () {58 vars in 1023 bytes} [Wed Jun 15 10:17:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 10:17:45.616180 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d58d41-cfe8-430f-9755-e08bc5751729 None None] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:17:45.635457 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc9361b-28dc-4491-b7fd-42091b900156 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:17:45.639371 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=100 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:17:45.669317 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6a08c1-e2ff-4a54-89ef-c97cc3dbdbc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:17:45.670127 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6a08c1-e2ff-4a54-89ef-c97cc3dbdbc8 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:17:45.670421 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6a08c1-e2ff-4a54-89ef-c97cc3dbdbc8 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:17:45.671203 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6a08c1-e2ff-4a54-89ef-c97cc3dbdbc8 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:17:45.683207 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c6a08c1-e2ff-4a54-89ef-c97cc3dbdbc8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 10:17:45.693177 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application [None req-4c6a08c1-e2ff-4a54-89ef-c97cc3dbdbc8 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 15 10:17:45.693177 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:17:45.693177 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:17:45.693177 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:17:45.693177 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:17:45.693177 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:17:45.693177 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:17:45.693177 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:17:45.693177 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:17:45.693177 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:17:45.693177 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:17:45.693177 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:17:45.693177 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 10:17:45.693177 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 10:17:45.693177 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 10:17:45.693177 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 10:17:45.694232 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 10:17:45.694232 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 10:17:45.694232 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 10:17:45.694232 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 10:17:45.694232 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:17:45.694232 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:17:45.694232 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:17:45.694232 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:17:45.694232 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:17:45.694232 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:17:45.694232 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:17:45.694232 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application with Lock( Jun 15 10:17:45.694232 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:17:45.694232 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:17:45.694232 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:17:45.694232 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:17:45.694999 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:17:45.694999 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:17:45.694999 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:17:45.694999 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:17:45.694999 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 10:17:45.694999 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 10:17:45.694999 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 10:17:45.694999 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 10:17:45.694999 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 10:17:45.694999 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 10:17:45.694999 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 15 10:17:45.694999 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Jun 15 10:17:45.694999 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 38/73] 10.222.0.49 () {56 vars in 1114 bytes} [Wed Jun 15 10:17:45 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:17:45.695914 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6a08c1-e2ff-4a54-89ef-c97cc3dbdbc8 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:17:45.727915 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d58d41-cfe8-430f-9755-e08bc5751729 None None] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:17:45.761424 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291e71d8-2b25-4709-ae31-9a5cc403f032 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:17:45.762097 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291e71d8-2b25-4709-ae31-9a5cc403f032 None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:17:45.762387 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291e71d8-2b25-4709-ae31-9a5cc403f032 None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:17:45.762669 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291e71d8-2b25-4709-ae31-9a5cc403f032 None admin] PATH_INFO: `/v3/roles` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:17:45.763440 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291e71d8-2b25-4709-ae31-9a5cc403f032 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:17:45.763748 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291e71d8-2b25-4709-ae31-9a5cc403f032 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:17:45.765145 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291e71d8-2b25-4709-ae31-9a5cc403f032 None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:17:45.775154 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 36/74] 10.222.0.49 () {56 vars in 1099 bytes} [Wed Jun 15 10:17:45 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:17:45.776573 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c3860c-db2f-415e-99d6-5375f0a8891c None None] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:17:45.776868 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c3860c-db2f-415e-99d6-5375f0a8891c None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:17:45.777146 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c3860c-db2f-415e-99d6-5375f0a8891c None None] PATH_INFO: `/` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:17:45.778386 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 37/75] 10.222.0.49 () {54 vars in 926 bytes} [Wed Jun 15 10:17:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:17:45.782468 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c3860c-db2f-415e-99d6-5375f0a8891c None None] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:17:45.800058 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6a08c1-e2ff-4a54-89ef-c97cc3dbdbc8 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:17:45.822536 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:17:45.834513 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc4d491-b791-4f0f-9e7f-b89cc4622797 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:17:45.835433 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc4d491-b791-4f0f-9e7f-b89cc4622797 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:17:45.835722 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc4d491-b791-4f0f-9e7f-b89cc4622797 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:17:45.836003 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc4d491-b791-4f0f-9e7f-b89cc4622797 None admin] PATH_INFO: `/v3/users/2f093b1bbc6148f39242d0692e1e8a95` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:17:45.851176 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc4d491-b791-4f0f-9e7f-b89cc4622797 None admin] RBAC: Authorizing `identity:get_user(user_id=2f093b1bbc6148f39242d0692e1e8a95)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:17:45.852634 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc4d491-b791-4f0f-9e7f-b89cc4622797 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:17:45.854301 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 39/76] 10.222.0.49 () {56 vars in 1177 bytes} [Wed Jun 15 10:17:45 2022] GET /identity/v3/users/2f093b1bbc6148f39242d0692e1e8a95 => generated 275 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:17:45.855066 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc4d491-b791-4f0f-9e7f-b89cc4622797 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:17:45.864855 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=100 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:17:45.885146 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c3860c-db2f-415e-99d6-5375f0a8891c None None] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:17:45.917854 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b36334-885d-4a21-a149-020e288d72f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:17:45.918595 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b36334-885d-4a21-a149-020e288d72f1 None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:17:45.918887 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b36334-885d-4a21-a149-020e288d72f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:17:45.919204 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b36334-885d-4a21-a149-020e288d72f1 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:17:45.930179 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b36334-885d-4a21-a149-020e288d72f1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 10:17:45.937838 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application [None req-99b36334-885d-4a21-a149-020e288d72f1 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 15 10:17:45.937838 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:17:45.937838 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:17:45.937838 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:17:45.937838 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:17:45.937838 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:17:45.937838 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:17:45.937838 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:17:45.937838 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:17:45.937838 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:17:45.937838 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:17:45.937838 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:17:45.937838 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 10:17:45.937838 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 10:17:45.937838 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 10:17:45.937838 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 10:17:45.938685 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 10:17:45.938685 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 10:17:45.938685 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 10:17:45.938685 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 10:17:45.938685 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:17:45.938685 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:17:45.938685 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:17:45.938685 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:17:45.938685 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:17:45.938685 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:17:45.938685 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:17:45.938685 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application with Lock( Jun 15 10:17:45.938685 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:17:45.938685 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:17:45.938685 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:17:45.938685 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:17:45.941200 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:17:45.941200 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:17:45.941200 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:17:45.941200 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:17:45.941200 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 10:17:45.941200 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 10:17:45.941200 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 10:17:45.941200 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 10:17:45.941200 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 10:17:45.941200 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 10:17:45.941200 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 15 10:17:45.941200 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application Jun 15 10:17:45.941200 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 38/77] 10.222.0.49 () {56 vars in 1099 bytes} [Wed Jun 15 10:17:45 2022] GET /identity/v3/roles/member => generated 84 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:17:45.941200 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b36334-885d-4a21-a149-020e288d72f1 None admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:17:45.954060 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc4d491-b791-4f0f-9e7f-b89cc4622797 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:17:45.989908 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eec7660-07e2-4901-a918-4fd7327b6b0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:17:45.990576 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eec7660-07e2-4901-a918-4fd7327b6b0f None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:17:45.990859 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eec7660-07e2-4901-a918-4fd7327b6b0f None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:17:45.991159 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eec7660-07e2-4901-a918-4fd7327b6b0f None admin] PATH_INFO: `/v3/roles` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:17:45.991911 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eec7660-07e2-4901-a918-4fd7327b6b0f None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:17:45.992255 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eec7660-07e2-4901-a918-4fd7327b6b0f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:17:45.993607 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eec7660-07e2-4901-a918-4fd7327b6b0f None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:17:46.001656 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 40/78] 10.222.0.49 () {56 vars in 1111 bytes} [Wed Jun 15 10:17:45 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:17:46.002772 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd87971-1535-4bf0-8d69-eb34b8826227 None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:17:46.003071 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd87971-1535-4bf0-8d69-eb34b8826227 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:17:46.003379 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd87971-1535-4bf0-8d69-eb34b8826227 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:17:46.020243 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-dbd87971-1535-4bf0-8d69-eb34b8826227 None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:17:46.021449 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-dbd87971-1535-4bf0-8d69-eb34b8826227 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:17:46.041668 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b36334-885d-4a21-a149-020e288d72f1 None admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:17:46.056581 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:17:46.075671 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49849bd4-0e88-4dc6-969b-8f5621e23c32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:17:46.076551 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49849bd4-0e88-4dc6-969b-8f5621e23c32 None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:17:46.076838 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49849bd4-0e88-4dc6-969b-8f5621e23c32 None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:17:46.077133 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49849bd4-0e88-4dc6-969b-8f5621e23c32 None admin] PATH_INFO: `/v3/users/fcdb2d8e71bd4fe6b9b1431183facba7` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:17:46.078361 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49849bd4-0e88-4dc6-969b-8f5621e23c32 None admin] RBAC: Authorizing `identity:get_user(user_id=fcdb2d8e71bd4fe6b9b1431183facba7)` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:17:46.079868 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49849bd4-0e88-4dc6-969b-8f5621e23c32 None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:17:46.081501 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 39/79] 10.222.0.49 () {56 vars in 1177 bytes} [Wed Jun 15 10:17:45 2022] GET /identity/v3/users/fcdb2d8e71bd4fe6b9b1431183facba7 => generated 247 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:17:46.081501 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49849bd4-0e88-4dc6-969b-8f5621e23c32 None admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:17:46.093942 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=100 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:17:46.097552 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 41/80] 10.222.0.49 () {58 vars in 1023 bytes} [Wed Jun 15 10:17:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 10:17:46.098414 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd87971-1535-4bf0-8d69-eb34b8826227 None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:17:46.183164 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49849bd4-0e88-4dc6-969b-8f5621e23c32 None admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:17:46.208050 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd87971-1535-4bf0-8d69-eb34b8826227 None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:17:46.222479 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca0a621-4ded-4cf2-8550-802157954cb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:17:46.223153 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca0a621-4ded-4cf2-8550-802157954cb0 None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:17:46.223443 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca0a621-4ded-4cf2-8550-802157954cb0 None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:17:46.223724 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca0a621-4ded-4cf2-8550-802157954cb0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:17:46.241199 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1869ca-2eb6-4e62-a979-fa7d810570f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:17:46.241875 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1869ca-2eb6-4e62-a979-fa7d810570f2 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:17:46.242165 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1869ca-2eb6-4e62-a979-fa7d810570f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:17:46.242452 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1869ca-2eb6-4e62-a979-fa7d810570f2 None admin] PATH_INFO: `/v3/roles` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:17:46.243248 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a1869ca-2eb6-4e62-a979-fa7d810570f2 None admin] RBAC: Adding query filter params (name=member) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:17:46.243564 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a1869ca-2eb6-4e62-a979-fa7d810570f2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:17:46.244961 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a1869ca-2eb6-4e62-a979-fa7d810570f2 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:17:46.252929 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 42/81] 10.222.0.49 () {56 vars in 1101 bytes} [Wed Jun 15 10:17:46 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:17:46.253713 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1869ca-2eb6-4e62-a979-fa7d810570f2 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:17:46.288722 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:17:46.291949 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca0a621-4ded-4cf2-8550-802157954cb0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:17:46.293421 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca0a621-4ded-4cf2-8550-802157954cb0 None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:17:46.304174 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=100 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:17:46.354184 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1869ca-2eb6-4e62-a979-fa7d810570f2 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:17:46.386802 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc123f98-c808-4636-b24c-1382973056aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:17:46.387698 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc123f98-c808-4636-b24c-1382973056aa None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:17:46.388004 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc123f98-c808-4636-b24c-1382973056aa None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:17:46.388286 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc123f98-c808-4636-b24c-1382973056aa None admin] PATH_INFO: `/v3/users/2f093b1bbc6148f39242d0692e1e8a95` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:17:46.390004 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc123f98-c808-4636-b24c-1382973056aa None admin] RBAC: Authorizing `identity:get_user(user_id=2f093b1bbc6148f39242d0692e1e8a95)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:17:46.391491 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc123f98-c808-4636-b24c-1382973056aa None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:17:46.396815 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 40/82] 10.222.0.49 () {58 vars in 1282 bytes} [Wed Jun 15 10:17:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 15 10:17:46.396815 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca0a621-4ded-4cf2-8550-802157954cb0 None admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:17:46.397018 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 43/83] 10.222.0.49 () {56 vars in 1177 bytes} [Wed Jun 15 10:17:46 2022] GET /identity/v3/users/2f093b1bbc6148f39242d0692e1e8a95 => generated 275 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:17:46.397018 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc123f98-c808-4636-b24c-1382973056aa None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:17:46.493149 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca0a621-4ded-4cf2-8550-802157954cb0 None admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:17:46.496385 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc123f98-c808-4636-b24c-1382973056aa None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:17:46.498032 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:17:46.503040 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=100 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:17:46.527688 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc4f92f-0060-4059-82e4-b20c48bfc1ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:17:46.528343 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc4f92f-0060-4059-82e4-b20c48bfc1ae None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:17:46.528630 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc4f92f-0060-4059-82e4-b20c48bfc1ae None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:17:46.528910 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc4f92f-0060-4059-82e4-b20c48bfc1ae None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:17:46.531684 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be8bf88-3be4-4c18-b8bf-207fd32b25ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:17:46.532479 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be8bf88-3be4-4c18-b8bf-207fd32b25ff None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:17:46.532767 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be8bf88-3be4-4c18-b8bf-207fd32b25ff None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:17:46.533048 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be8bf88-3be4-4c18-b8bf-207fd32b25ff None admin] PATH_INFO: `/v3/domains/default` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:17:46.534224 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be8bf88-3be4-4c18-b8bf-207fd32b25ff None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:17:46.535725 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be8bf88-3be4-4c18-b8bf-207fd32b25ff None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:17:46.537354 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 44/84] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Jun 15 10:17:46 2022] GET /identity/v3/domains/default => generated 199 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:17:46.538206 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be8bf88-3be4-4c18-b8bf-207fd32b25ff None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:17:46.597238 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc4f92f-0060-4059-82e4-b20c48bfc1ae None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:17:46.598737 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc4f92f-0060-4059-82e4-b20c48bfc1ae None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:17:46.635758 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be8bf88-3be4-4c18-b8bf-207fd32b25ff None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:17:46.668578 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b9402e-baef-46b4-a2d6-6636a223d411 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:17:46.669443 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b9402e-baef-46b4-a2d6-6636a223d411 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:17:46.669737 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b9402e-baef-46b4-a2d6-6636a223d411 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:17:46.670020 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b9402e-baef-46b4-a2d6-6636a223d411 None admin] PATH_INFO: `/v3/users/2f093b1bbc6148f39242d0692e1e8a95` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:17:46.671819 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b9402e-baef-46b4-a2d6-6636a223d411 None admin] RBAC: Authorizing `identity:get_user(user_id=2f093b1bbc6148f39242d0692e1e8a95)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:17:46.673317 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b9402e-baef-46b4-a2d6-6636a223d411 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:17:46.674946 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 45/85] 10.222.0.49 () {56 vars in 1177 bytes} [Wed Jun 15 10:17:46 2022] GET /identity/v3/users/2f093b1bbc6148f39242d0692e1e8a95 => generated 275 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:17:46.680054 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b9402e-baef-46b4-a2d6-6636a223d411 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:17:46.696601 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 41/86] 10.222.0.49 () {58 vars in 1282 bytes} [Wed Jun 15 10:17:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 15 10:17:46.700023 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc4f92f-0060-4059-82e4-b20c48bfc1ae None admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:17:46.710460 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=100 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:17:46.719492 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:17:46.778272 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b9402e-baef-46b4-a2d6-6636a223d411 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:17:46.798697 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc4f92f-0060-4059-82e4-b20c48bfc1ae None admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:17:46.812850 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13470fd-b20f-49b4-92ad-3285572373d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:17:46.813718 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13470fd-b20f-49b4-92ad-3285572373d1 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:17:46.813997 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13470fd-b20f-49b4-92ad-3285572373d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:17:46.814267 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13470fd-b20f-49b4-92ad-3285572373d1 None admin] PATH_INFO: `/v3/projects/5f3cd857995e418386bb22cdf90c1ae9` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:17:46.815471 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c13470fd-b20f-49b4-92ad-3285572373d1 None admin] RBAC: Authorizing `identity:get_project(project_id=5f3cd857995e418386bb22cdf90c1ae9)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:17:46.816913 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c13470fd-b20f-49b4-92ad-3285572373d1 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:17:46.818601 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 46/87] 10.222.0.49 () {56 vars in 1186 bytes} [Wed Jun 15 10:17:46 2022] GET /identity/v3/projects/5f3cd857995e418386bb22cdf90c1ae9 => generated 298 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:17:46.819385 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13470fd-b20f-49b4-92ad-3285572373d1 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:17:46.832027 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2cc77c6-1ee0-46f1-84a6-eec0f6e0f555 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:17:46.832680 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cc77c6-1ee0-46f1-84a6-eec0f6e0f555 None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:17:46.832968 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cc77c6-1ee0-46f1-84a6-eec0f6e0f555 None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:17:46.833248 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cc77c6-1ee0-46f1-84a6-eec0f6e0f555 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:17:46.897786 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2cc77c6-1ee0-46f1-84a6-eec0f6e0f555 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:17:46.899289 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2cc77c6-1ee0-46f1-84a6-eec0f6e0f555 None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:17:46.908986 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=100 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:17:46.922566 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13470fd-b20f-49b4-92ad-3285572373d1 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:17:46.930038 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:17:46.956809 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6922a859-e870-48bf-971a-bf1ed01a2382 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:17:46.957481 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6922a859-e870-48bf-971a-bf1ed01a2382 None admin] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:17:46.957764 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6922a859-e870-48bf-971a-bf1ed01a2382 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:17:46.958041 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6922a859-e870-48bf-971a-bf1ed01a2382 None admin] PATH_INFO: `/v3/users` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:17:46.958749 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6922a859-e870-48bf-971a-bf1ed01a2382 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:17:46.960218 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6922a859-e870-48bf-971a-bf1ed01a2382 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:17:46.979498 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 47/88] 10.222.0.49 () {60 vars in 1132 bytes} [Wed Jun 15 10:17:46 2022] POST /identity/v3/users => generated 289 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 10:17:46.980246 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6922a859-e870-48bf-971a-bf1ed01a2382 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:17:47.010418 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 42/89] 10.222.0.49 () {58 vars in 1282 bytes} [Wed Jun 15 10:17:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 15 10:17:47.010418 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2cc77c6-1ee0-46f1-84a6-eec0f6e0f555 None admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:17:47.097601 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6922a859-e870-48bf-971a-bf1ed01a2382 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:17:47.121067 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2cc77c6-1ee0-46f1-84a6-eec0f6e0f555 None admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:17:47.131840 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=100 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:17:47.135740 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0394ac92-7c6a-42ba-b121-335d7abd48e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:17:47.136411 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0394ac92-7c6a-42ba-b121-335d7abd48e6 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:17:47.136568 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0394ac92-7c6a-42ba-b121-335d7abd48e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:17:47.136842 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0394ac92-7c6a-42ba-b121-335d7abd48e6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:17:47.162249 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25e601b-698e-41ae-ab45-dbce2835e37b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:17:47.163146 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25e601b-698e-41ae-ab45-dbce2835e37b None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:17:47.163438 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25e601b-698e-41ae-ab45-dbce2835e37b None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:17:47.163718 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25e601b-698e-41ae-ab45-dbce2835e37b None admin] PATH_INFO: `/v3/projects/5f3cd857995e418386bb22cdf90c1ae9` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:17:47.164903 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25e601b-698e-41ae-ab45-dbce2835e37b None admin] RBAC: Authorizing `identity:get_project(project_id=5f3cd857995e418386bb22cdf90c1ae9)` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:17:47.166451 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25e601b-698e-41ae-ab45-dbce2835e37b None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:17:47.168941 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 43/90] 10.222.0.49 () {56 vars in 1186 bytes} [Wed Jun 15 10:17:47 2022] GET /identity/v3/projects/5f3cd857995e418386bb22cdf90c1ae9 => generated 298 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:17:47.169054 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25e601b-698e-41ae-ab45-dbce2835e37b None admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:17:47.179452 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:17:47.228677 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0394ac92-7c6a-42ba-b121-335d7abd48e6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:17:47.230171 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0394ac92-7c6a-42ba-b121-335d7abd48e6 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:17:47.293586 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25e601b-698e-41ae-ab45-dbce2835e37b None admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:17:47.334250 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66de159-f6db-455e-95e9-79c8ef14e8eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:17:47.334923 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66de159-f6db-455e-95e9-79c8ef14e8eb None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:17:47.335241 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66de159-f6db-455e-95e9-79c8ef14e8eb None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:17:47.335520 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66de159-f6db-455e-95e9-79c8ef14e8eb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:17:47.336355 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d66de159-f6db-455e-95e9-79c8ef14e8eb None admin] RBAC: Adding query filter params (role.id=6876e57cc51c448097ab109d35b2a3da, scope.project.id=5f3cd857995e418386bb22cdf90c1ae9, user.id=2f093b1bbc6148f39242d0692e1e8a95) {{(pid=80119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:17:47.336668 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d66de159-f6db-455e-95e9-79c8ef14e8eb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:17:47.338132 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d66de159-f6db-455e-95e9-79c8ef14e8eb None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:17:47.344031 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 44/91] 10.222.0.49 () {56 vars in 1374 bytes} [Wed Jun 15 10:17:47 2022] GET /identity/v3/role_assignments?user.id=2f093b1bbc6148f39242d0692e1e8a95&scope.project.id=5f3cd857995e418386bb22cdf90c1ae9&role.id=6876e57cc51c448097ab109d35b2a3da => generated 259 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:17:47.345138 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66de159-f6db-455e-95e9-79c8ef14e8eb None admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:17:47.368326 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 48/92] 10.222.0.49 () {58 vars in 1282 bytes} [Wed Jun 15 10:17:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 388 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 15 10:17:47.369143 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0394ac92-7c6a-42ba-b121-335d7abd48e6 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:17:47.427852 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=100 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:17:47.498028 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:17:47.540968 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66de159-f6db-455e-95e9-79c8ef14e8eb None admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:17:47.567315 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0394ac92-7c6a-42ba-b121-335d7abd48e6 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:17:47.602631 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e267f7cc-e8fe-4b6b-9dee-ee3553402f02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:17:47.603532 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e267f7cc-e8fe-4b6b-9dee-ee3553402f02 None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:17:47.603815 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e267f7cc-e8fe-4b6b-9dee-ee3553402f02 None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:17:47.604090 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e267f7cc-e8fe-4b6b-9dee-ee3553402f02 None admin] PATH_INFO: `/v3/projects/5f3cd857995e418386bb22cdf90c1ae9` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:17:47.605703 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e267f7cc-e8fe-4b6b-9dee-ee3553402f02 None admin] RBAC: Authorizing `identity:get_project(project_id=5f3cd857995e418386bb22cdf90c1ae9)` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:17:47.607194 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e267f7cc-e8fe-4b6b-9dee-ee3553402f02 None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:17:47.609463 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 45/93] 10.222.0.49 () {56 vars in 1186 bytes} [Wed Jun 15 10:17:47 2022] GET /identity/v3/projects/5f3cd857995e418386bb22cdf90c1ae9 => generated 298 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:17:47.610231 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e267f7cc-e8fe-4b6b-9dee-ee3553402f02 None admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:17:47.627280 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff01e56-9999-440e-bd44-413696a4101d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:17:47.627873 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff01e56-9999-440e-bd44-413696a4101d None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:17:47.628158 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff01e56-9999-440e-bd44-413696a4101d None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:17:47.628434 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff01e56-9999-440e-bd44-413696a4101d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:17:47.629272 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff01e56-9999-440e-bd44-413696a4101d None admin] RBAC: Adding query filter params (role.id=8051174e7a7947d5b991ebd793c2e543, scope.project.id=5f3cd857995e418386bb22cdf90c1ae9, user.id=fcdb2d8e71bd4fe6b9b1431183facba7) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:17:47.629583 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff01e56-9999-440e-bd44-413696a4101d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:17:47.631039 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff01e56-9999-440e-bd44-413696a4101d None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:17:47.640062 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 49/94] 10.222.0.49 () {56 vars in 1374 bytes} [Wed Jun 15 10:17:47 2022] GET /identity/v3/role_assignments?user.id=fcdb2d8e71bd4fe6b9b1431183facba7&scope.project.id=5f3cd857995e418386bb22cdf90c1ae9&role.id=8051174e7a7947d5b991ebd793c2e543 => generated 259 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:17:47.640723 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff01e56-9999-440e-bd44-413696a4101d None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:17:47.829798 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e267f7cc-e8fe-4b6b-9dee-ee3553402f02 None admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:17:47.853736 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=100 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:17:47.890272 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df09b4fb-40f5-4605-8ebe-87b67386540d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:17:47.891351 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df09b4fb-40f5-4605-8ebe-87b67386540d None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:17:47.891638 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df09b4fb-40f5-4605-8ebe-87b67386540d None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:17:47.891915 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df09b4fb-40f5-4605-8ebe-87b67386540d None admin] PATH_INFO: `/v3/projects/24ea2a1e0d1f49bebb8a85e96ebbe085` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:17:47.893247 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df09b4fb-40f5-4605-8ebe-87b67386540d None admin] RBAC: Authorizing `identity:get_project(project_id=24ea2a1e0d1f49bebb8a85e96ebbe085)` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:17:47.894719 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df09b4fb-40f5-4605-8ebe-87b67386540d None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:17:47.897131 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 46/95] 10.222.0.49 () {56 vars in 1186 bytes} [Wed Jun 15 10:17:47 2022] GET /identity/v3/projects/24ea2a1e0d1f49bebb8a85e96ebbe085 => generated 312 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:17:47.902771 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df09b4fb-40f5-4605-8ebe-87b67386540d None admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:17:47.903957 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff01e56-9999-440e-bd44-413696a4101d None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:17:47.961344 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:17:47.979945 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a75e91-5f4a-48bc-bccf-fef430225b6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:17:47.980868 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a75e91-5f4a-48bc-bccf-fef430225b6b None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:17:47.981295 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a75e91-5f4a-48bc-bccf-fef430225b6b None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:17:47.981727 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a75e91-5f4a-48bc-bccf-fef430225b6b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:17:47.982703 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a75e91-5f4a-48bc-bccf-fef430225b6b None admin] RBAC: Adding query filter params (role.id=4d2118569d074a288b37d120110c300d, scope.project.id=5f3cd857995e418386bb22cdf90c1ae9, user.id=2f093b1bbc6148f39242d0692e1e8a95) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:17:47.983716 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a75e91-5f4a-48bc-bccf-fef430225b6b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:17:47.986331 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a75e91-5f4a-48bc-bccf-fef430225b6b None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:17:47.997408 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 50/96] 10.222.0.49 () {56 vars in 1374 bytes} [Wed Jun 15 10:17:47 2022] GET /identity/v3/role_assignments?user.id=2f093b1bbc6148f39242d0692e1e8a95&scope.project.id=5f3cd857995e418386bb22cdf90c1ae9&role.id=4d2118569d074a288b37d120110c300d => generated 259 bytes in 357 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:17:48.059007 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df09b4fb-40f5-4605-8ebe-87b67386540d None admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:17:48.108951 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a3308a-4b9a-4a27-bd8f-5e823a064df8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:17:48.110520 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a3308a-4b9a-4a27-bd8f-5e823a064df8 None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:17:48.111266 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a3308a-4b9a-4a27-bd8f-5e823a064df8 None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:17:48.111980 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a3308a-4b9a-4a27-bd8f-5e823a064df8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:17:48.113628 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a3308a-4b9a-4a27-bd8f-5e823a064df8 None admin] RBAC: Adding query filter params (role.id=6876e57cc51c448097ab109d35b2a3da, scope.project.id=24ea2a1e0d1f49bebb8a85e96ebbe085, user.id=2f093b1bbc6148f39242d0692e1e8a95) {{(pid=80119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:17:48.114344 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a3308a-4b9a-4a27-bd8f-5e823a064df8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:17:48.116166 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a3308a-4b9a-4a27-bd8f-5e823a064df8 None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:17:48.122679 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 47/97] 10.222.0.49 () {56 vars in 1374 bytes} [Wed Jun 15 10:17:47 2022] GET /identity/v3/role_assignments?user.id=2f093b1bbc6148f39242d0692e1e8a95&scope.project.id=24ea2a1e0d1f49bebb8a85e96ebbe085&role.id=6876e57cc51c448097ab109d35b2a3da => generated 259 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:17:49.700855 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d8b2d4-9f6e-452a-80c7-945df2e0ba72 None None] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:17:49.701365 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d8b2d4-9f6e-452a-80c7-945df2e0ba72 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:17:49.701439 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d8b2d4-9f6e-452a-80c7-945df2e0ba72 None None] PATH_INFO: `/` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:17:49.703642 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 51/98] 10.222.0.49 () {54 vars in 926 bytes} [Wed Jun 15 10:17:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:17:49.708862 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ba4d0e-b2fa-4607-8d1e-8f5bfb7ff1db None None] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:17:49.709674 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ba4d0e-b2fa-4607-8d1e-8f5bfb7ff1db None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:17:49.710106 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ba4d0e-b2fa-4607-8d1e-8f5bfb7ff1db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:17:49.728513 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.auth.core [None req-22ba4d0e-b2fa-4607-8d1e-8f5bfb7ff1db None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:17:49.729995 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-22ba4d0e-b2fa-4607-8d1e-8f5bfb7ff1db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:17:49.775910 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=100 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:17:49.825098 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 48/99] 10.222.0.49 () {58 vars in 1023 bytes} [Wed Jun 15 10:17:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 10:17:49.861205 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d8b2d4-9f6e-452a-80c7-945df2e0ba72 None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:17:49.992839 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d8b2d4-9f6e-452a-80c7-945df2e0ba72 None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:17:50.038610 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce70858-96f2-4c19-aad2-551e1ea039a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:17:50.039460 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce70858-96f2-4c19-aad2-551e1ea039a8 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:17:50.039754 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce70858-96f2-4c19-aad2-551e1ea039a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:17:50.040039 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce70858-96f2-4c19-aad2-551e1ea039a8 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:17:50.056561 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce70858-96f2-4c19-aad2-551e1ea039a8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 10:17:50.064928 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application [None req-dce70858-96f2-4c19-aad2-551e1ea039a8 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 15 10:17:50.064928 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:17:50.064928 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:17:50.064928 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:17:50.064928 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:17:50.064928 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:17:50.064928 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:17:50.064928 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:17:50.064928 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:17:50.064928 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:17:50.064928 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:17:50.064928 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:17:50.064928 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 10:17:50.064928 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 10:17:50.064928 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 10:17:50.064928 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 10:17:50.065969 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 10:17:50.065969 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 10:17:50.065969 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 10:17:50.065969 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 10:17:50.065969 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:17:50.065969 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:17:50.065969 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:17:50.065969 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:17:50.065969 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:17:50.065969 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:17:50.065969 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:17:50.065969 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application with Lock( Jun 15 10:17:50.065969 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:17:50.065969 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:17:50.065969 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:17:50.065969 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:17:50.066836 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:17:50.066836 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:17:50.066836 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:17:50.066836 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:17:50.066836 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 10:17:50.066836 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 10:17:50.066836 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 10:17:50.066836 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 10:17:50.066836 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 10:17:50.066836 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 10:17:50.066836 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 15 10:17:50.066836 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Jun 15 10:17:50.066836 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 52/100] 10.222.0.49 () {56 vars in 1099 bytes} [Wed Jun 15 10:17:49 2022] GET /identity/v3/roles/reader => generated 84 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:17:50.071811 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ba4d0e-b2fa-4607-8d1e-8f5bfb7ff1db None None] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:17:50.218150 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ba4d0e-b2fa-4607-8d1e-8f5bfb7ff1db None None] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:17:50.264333 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c59f7e-a1fa-42b7-b4ed-25bab5b41997 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:17:50.265496 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c59f7e-a1fa-42b7-b4ed-25bab5b41997 None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:17:50.265789 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c59f7e-a1fa-42b7-b4ed-25bab5b41997 None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:17:50.266072 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c59f7e-a1fa-42b7-b4ed-25bab5b41997 None admin] PATH_INFO: `/v3/roles` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:17:50.266849 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c59f7e-a1fa-42b7-b4ed-25bab5b41997 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:17:50.267187 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c59f7e-a1fa-42b7-b4ed-25bab5b41997 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:17:50.268627 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c59f7e-a1fa-42b7-b4ed-25bab5b41997 None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:17:50.283408 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 49/101] 10.222.0.49 () {56 vars in 1101 bytes} [Wed Jun 15 10:17:50 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:17:50.288838 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce70858-96f2-4c19-aad2-551e1ea039a8 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:17:50.317316 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d72639-0b27-4a2f-a212-aa865e335a6e None None] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:17:50.317612 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d72639-0b27-4a2f-a212-aa865e335a6e None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:17:50.317900 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d72639-0b27-4a2f-a212-aa865e335a6e None None] PATH_INFO: `/` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:17:50.319338 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 50/102] 10.222.0.49 () {54 vars in 926 bytes} [Wed Jun 15 10:17:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 15 10:17:50.337084 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5e967d-b94a-4091-add3-976c61ccec00 None None] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:17:50.337374 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5e967d-b94a-4091-add3-976c61ccec00 None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:17:50.337661 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5e967d-b94a-4091-add3-976c61ccec00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:17:50.365461 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.auth.core [None req-8d5e967d-b94a-4091-add3-976c61ccec00 None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:17:50.366707 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-8d5e967d-b94a-4091-add3-976c61ccec00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:17:50.387470 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:17:50.427437 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=100 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:17:50.481018 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce70858-96f2-4c19-aad2-551e1ea039a8 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:17:50.487760 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 51/103] 10.222.0.49 () {58 vars in 1023 bytes} [Wed Jun 15 10:17:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 10:17:50.489091 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae17dd3-d213-42b6-99aa-86e2007823ce None None] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:17:50.489384 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae17dd3-d213-42b6-99aa-86e2007823ce None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:17:50.489664 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae17dd3-d213-42b6-99aa-86e2007823ce None None] PATH_INFO: `/` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:17:50.490924 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 52/104] 10.222.0.49 () {54 vars in 926 bytes} [Wed Jun 15 10:17:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:17:50.501060 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2627d37-da83-4818-96a3-713f12fdf61c None None] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:17:50.501345 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2627d37-da83-4818-96a3-713f12fdf61c None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:17:50.501627 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2627d37-da83-4818-96a3-713f12fdf61c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:17:50.540147 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.auth.core [None req-a2627d37-da83-4818-96a3-713f12fdf61c None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:17:50.541380 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-a2627d37-da83-4818-96a3-713f12fdf61c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:17:50.543289 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526be38d-927e-436b-a13f-3bafb22e4f37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:17:50.544176 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526be38d-927e-436b-a13f-3bafb22e4f37 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:17:50.544459 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526be38d-927e-436b-a13f-3bafb22e4f37 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:17:50.544736 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526be38d-927e-436b-a13f-3bafb22e4f37 None admin] PATH_INFO: `/v3/users/dfe1d4745a4d4642a21f8860b4a02fb2` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:17:50.561797 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526be38d-927e-436b-a13f-3bafb22e4f37 None admin] RBAC: Authorizing `identity:get_user(user_id=dfe1d4745a4d4642a21f8860b4a02fb2)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:17:50.565668 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526be38d-927e-436b-a13f-3bafb22e4f37 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:17:50.567423 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 53/105] 10.222.0.49 () {56 vars in 1177 bytes} [Wed Jun 15 10:17:50 2022] GET /identity/v3/users/dfe1d4745a4d4642a21f8860b4a02fb2 => generated 289 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:17:50.568186 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526be38d-927e-436b-a13f-3bafb22e4f37 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:17:50.667942 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 53/106] 10.222.0.49 () {58 vars in 1023 bytes} [Wed Jun 15 10:17:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 10:17:50.669021 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2627d37-da83-4818-96a3-713f12fdf61c None None] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:17:50.789063 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526be38d-927e-436b-a13f-3bafb22e4f37 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:17:50.807839 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:17:50.859726 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87d91b7-d64d-418d-9498-c18988f885a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:17:50.860611 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87d91b7-d64d-418d-9498-c18988f885a9 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:17:50.860902 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87d91b7-d64d-418d-9498-c18988f885a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:17:50.861181 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87d91b7-d64d-418d-9498-c18988f885a9 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:17:50.867487 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=100 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:17:50.883853 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87d91b7-d64d-418d-9498-c18988f885a9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 10:17:50.888920 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2627d37-da83-4818-96a3-713f12fdf61c None None] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:17:50.901849 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application [None req-d87d91b7-d64d-418d-9498-c18988f885a9 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 15 10:17:50.901849 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:17:50.901849 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:17:50.901849 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:17:50.901849 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:17:50.901849 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:17:50.901849 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:17:50.901849 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:17:50.901849 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:17:50.901849 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:17:50.901849 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:17:50.901849 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:17:50.901849 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 10:17:50.901849 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 10:17:50.901849 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 10:17:50.901849 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 10:17:50.902841 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 10:17:50.902841 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 10:17:50.902841 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 10:17:50.902841 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 10:17:50.902841 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:17:50.902841 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:17:50.902841 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:17:50.902841 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:17:50.902841 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:17:50.902841 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:17:50.902841 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:17:50.902841 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application with Lock( Jun 15 10:17:50.902841 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:17:50.902841 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:17:50.902841 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:17:50.902841 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:17:50.904622 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:17:50.904622 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:17:50.904622 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:17:50.904622 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:17:50.904622 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 10:17:50.904622 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 10:17:50.904622 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 10:17:50.904622 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 10:17:50.904622 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 10:17:50.904622 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 10:17:50.904622 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 15 10:17:50.904622 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Jun 15 10:17:50.904622 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 54/107] 10.222.0.49 () {56 vars in 1099 bytes} [Wed Jun 15 10:17:50 2022] GET /identity/v3/roles/member => generated 84 bytes in 335 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:17:50.904622 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87d91b7-d64d-418d-9498-c18988f885a9 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:17:50.948437 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5861c63-a6be-459b-a143-f73fbfd5edde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:17:50.949108 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5861c63-a6be-459b-a143-f73fbfd5edde None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:17:50.949402 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5861c63-a6be-459b-a143-f73fbfd5edde None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:17:50.949687 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5861c63-a6be-459b-a143-f73fbfd5edde None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:17:51.031587 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87d91b7-d64d-418d-9498-c18988f885a9 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:17:51.036267 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5861c63-a6be-459b-a143-f73fbfd5edde None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:17:51.037779 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5861c63-a6be-459b-a143-f73fbfd5edde None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:17:51.067707 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c8a066-50af-4362-aabb-7cf37caf9983 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:17:51.068805 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c8a066-50af-4362-aabb-7cf37caf9983 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:17:51.069280 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c8a066-50af-4362-aabb-7cf37caf9983 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:17:51.069736 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c8a066-50af-4362-aabb-7cf37caf9983 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:17:51.071172 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c8a066-50af-4362-aabb-7cf37caf9983 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:17:51.072860 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c8a066-50af-4362-aabb-7cf37caf9983 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:17:51.074671 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 55/108] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Jun 15 10:17:50 2022] GET /identity/v3/domains/default => generated 199 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:17:51.075785 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c8a066-50af-4362-aabb-7cf37caf9983 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:17:51.113382 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:17:51.171550 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=100 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:17:51.190243 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 54/109] 10.222.0.49 () {58 vars in 1282 bytes} [Wed Jun 15 10:17:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 517 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 15 10:17:51.190243 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2daecd-12ae-40ec-9900-2e8d16fa9eee None None] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:17:51.190243 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2daecd-12ae-40ec-9900-2e8d16fa9eee None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:17:51.190243 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2daecd-12ae-40ec-9900-2e8d16fa9eee None None] PATH_INFO: `/` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:17:51.190243 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 55/110] 10.222.0.49 () {54 vars in 926 bytes} [Wed Jun 15 10:17:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:17:51.191214 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2daecd-12ae-40ec-9900-2e8d16fa9eee None None] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:17:51.201604 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c8a066-50af-4362-aabb-7cf37caf9983 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:17:51.236050 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40286cc8-0802-49ef-9f08-04ea3d9402dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:17:51.236730 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40286cc8-0802-49ef-9f08-04ea3d9402dc None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:17:51.237020 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40286cc8-0802-49ef-9f08-04ea3d9402dc None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:17:51.237300 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40286cc8-0802-49ef-9f08-04ea3d9402dc None admin] PATH_INFO: `/v3/roles` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:17:51.238056 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40286cc8-0802-49ef-9f08-04ea3d9402dc None admin] RBAC: Adding query filter params (name=member) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:17:51.238368 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40286cc8-0802-49ef-9f08-04ea3d9402dc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:17:51.239790 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40286cc8-0802-49ef-9f08-04ea3d9402dc None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:17:51.247885 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 56/111] 10.222.0.49 () {56 vars in 1101 bytes} [Wed Jun 15 10:17:51 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:17:51.249280 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e8f51a-ec1b-414b-9d9e-b21169d0a8a0 None None] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:17:51.249569 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e8f51a-ec1b-414b-9d9e-b21169d0a8a0 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:17:51.249849 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e8f51a-ec1b-414b-9d9e-b21169d0a8a0 None None] PATH_INFO: `/` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:17:51.251074 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 57/112] 10.222.0.49 () {54 vars in 926 bytes} [Wed Jun 15 10:17:51 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:17:51.251796 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e8f51a-ec1b-414b-9d9e-b21169d0a8a0 None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:17:51.295781 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2daecd-12ae-40ec-9900-2e8d16fa9eee None None] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:17:51.329791 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19fe004-376c-4562-b714-da0c5250167b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:17:51.330387 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19fe004-376c-4562-b714-da0c5250167b None admin] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:17:51.330678 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19fe004-376c-4562-b714-da0c5250167b None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:17:51.330959 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19fe004-376c-4562-b714-da0c5250167b None admin] PATH_INFO: `/v3/projects` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:17:51.331695 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e19fe004-376c-4562-b714-da0c5250167b None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:17:51.333163 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e19fe004-376c-4562-b714-da0c5250167b None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:17:51.347232 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:17:51.352472 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e8f51a-ec1b-414b-9d9e-b21169d0a8a0 None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:17:51.359578 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 56/113] 10.222.0.49 () {60 vars in 1141 bytes} [Wed Jun 15 10:17:51 2022] POST /identity/v3/projects => generated 302 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 10:17:51.359578 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939f30be-d561-4ef3-af8f-61212c1a0900 None None] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:17:51.359578 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939f30be-d561-4ef3-af8f-61212c1a0900 None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:17:51.359578 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939f30be-d561-4ef3-af8f-61212c1a0900 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:17:51.387282 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274fbddd-5006-423f-b58e-1830e6f74664 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:17:51.388186 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274fbddd-5006-423f-b58e-1830e6f74664 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:17:51.388473 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274fbddd-5006-423f-b58e-1830e6f74664 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:17:51.388746 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274fbddd-5006-423f-b58e-1830e6f74664 None admin] PATH_INFO: `/v3/projects/5f3cd857995e418386bb22cdf90c1ae9` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:17:51.390554 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-274fbddd-5006-423f-b58e-1830e6f74664 None admin] RBAC: Authorizing `identity:get_project(project_id=5f3cd857995e418386bb22cdf90c1ae9)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:17:51.392165 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-274fbddd-5006-423f-b58e-1830e6f74664 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:17:51.395779 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.auth.core [None req-939f30be-d561-4ef3-af8f-61212c1a0900 None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:17:51.397045 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-939f30be-d561-4ef3-af8f-61212c1a0900 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:17:51.399161 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 58/114] 10.222.0.49 () {56 vars in 1186 bytes} [Wed Jun 15 10:17:51 2022] GET /identity/v3/projects/5f3cd857995e418386bb22cdf90c1ae9 => generated 298 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:17:51.400781 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8ef7b3-1a33-4fe4-911d-3dd5ad30ddc8 None None] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:17:51.401072 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8ef7b3-1a33-4fe4-911d-3dd5ad30ddc8 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:17:51.401349 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8ef7b3-1a33-4fe4-911d-3dd5ad30ddc8 None None] PATH_INFO: `/` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:17:51.403014 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 59/115] 10.222.0.49 () {54 vars in 926 bytes} [Wed Jun 15 10:17:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:17:51.403786 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8ef7b3-1a33-4fe4-911d-3dd5ad30ddc8 None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:17:51.447884 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:17:51.482805 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 57/116] 10.222.0.49 () {58 vars in 1023 bytes} [Wed Jun 15 10:17:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 10:17:51.484517 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec515877-7ebb-4eef-9492-f28d5872a725 None None] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:17:51.484866 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec515877-7ebb-4eef-9492-f28d5872a725 None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:17:51.485249 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec515877-7ebb-4eef-9492-f28d5872a725 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:17:51.507075 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.auth.core [None req-ec515877-7ebb-4eef-9492-f28d5872a725 None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:17:51.508459 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-ec515877-7ebb-4eef-9492-f28d5872a725 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:17:51.539350 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8ef7b3-1a33-4fe4-911d-3dd5ad30ddc8 None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:17:51.579062 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3772c0ea-8f28-494a-bb4f-fece813d2dc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:17:51.580139 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3772c0ea-8f28-494a-bb4f-fece813d2dc5 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:17:51.580366 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3772c0ea-8f28-494a-bb4f-fece813d2dc5 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:17:51.580649 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3772c0ea-8f28-494a-bb4f-fece813d2dc5 None admin] PATH_INFO: `/v3/users/2f093b1bbc6148f39242d0692e1e8a95` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:17:51.581864 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3772c0ea-8f28-494a-bb4f-fece813d2dc5 None admin] RBAC: Authorizing `identity:get_user(user_id=2f093b1bbc6148f39242d0692e1e8a95)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:17:51.583361 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3772c0ea-8f28-494a-bb4f-fece813d2dc5 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:17:51.585008 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 60/117] 10.222.0.49 () {56 vars in 1177 bytes} [Wed Jun 15 10:17:51 2022] GET /identity/v3/users/2f093b1bbc6148f39242d0692e1e8a95 => generated 275 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:17:51.585503 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 58/118] 10.222.0.49 () {58 vars in 1023 bytes} [Wed Jun 15 10:17:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 10:17:51.585782 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3772c0ea-8f28-494a-bb4f-fece813d2dc5 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:17:51.586683 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfbb397-9060-4ed2-b4cf-8f57088ef321 None None] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:17:51.586977 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfbb397-9060-4ed2-b4cf-8f57088ef321 None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:17:51.591548 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfbb397-9060-4ed2-b4cf-8f57088ef321 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:17:51.609029 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.auth.core [None req-6bfbb397-9060-4ed2-b4cf-8f57088ef321 None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:17:51.610231 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-6bfbb397-9060-4ed2-b4cf-8f57088ef321 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:17:51.626978 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:17:51.681705 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 59/119] 10.222.0.49 () {58 vars in 1023 bytes} [Wed Jun 15 10:17:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 10:17:51.682550 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bfbb397-9060-4ed2-b4cf-8f57088ef321 None None] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:17:51.708685 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3772c0ea-8f28-494a-bb4f-fece813d2dc5 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:17:51.719422 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=100 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:17:51.748301 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51dad9c4-7d1a-4fc8-85e6-41cf816bf5fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:17:51.749499 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51dad9c4-7d1a-4fc8-85e6-41cf816bf5fd None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:17:51.749787 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51dad9c4-7d1a-4fc8-85e6-41cf816bf5fd None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:17:51.750133 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51dad9c4-7d1a-4fc8-85e6-41cf816bf5fd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:17:51.750886 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51dad9c4-7d1a-4fc8-85e6-41cf816bf5fd None admin] RBAC: Adding query filter params (role.id=301a3c8a23ba40edbc35df1da8af0de9, scope.project.id=5f3cd857995e418386bb22cdf90c1ae9, user.id=dfe1d4745a4d4642a21f8860b4a02fb2) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:17:51.751231 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51dad9c4-7d1a-4fc8-85e6-41cf816bf5fd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:17:51.752674 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51dad9c4-7d1a-4fc8-85e6-41cf816bf5fd None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:17:51.758310 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 61/120] 10.222.0.49 () {56 vars in 1374 bytes} [Wed Jun 15 10:17:51 2022] GET /identity/v3/role_assignments?user.id=dfe1d4745a4d4642a21f8860b4a02fb2&scope.project.id=5f3cd857995e418386bb22cdf90c1ae9&role.id=301a3c8a23ba40edbc35df1da8af0de9 => generated 259 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:17:51.764202 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51dad9c4-7d1a-4fc8-85e6-41cf816bf5fd None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:17:51.817727 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bfbb397-9060-4ed2-b4cf-8f57088ef321 None None] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:17:51.863922 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f3d8d0-8281-4f9a-8a8b-99e5f604a19f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:17:51.865152 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f3d8d0-8281-4f9a-8a8b-99e5f604a19f None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:17:51.865705 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f3d8d0-8281-4f9a-8a8b-99e5f604a19f None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:17:51.866191 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f3d8d0-8281-4f9a-8a8b-99e5f604a19f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:17:51.879228 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f3d8d0-8281-4f9a-8a8b-99e5f604a19f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 10:17:51.887998 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application [None req-e2f3d8d0-8281-4f9a-8a8b-99e5f604a19f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 10:17:51.887998 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:17:51.887998 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:17:51.887998 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:17:51.887998 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:17:51.887998 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:17:51.887998 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:17:51.887998 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:17:51.887998 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:17:51.887998 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:17:51.887998 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:17:51.887998 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:17:51.887998 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 10:17:51.887998 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 10:17:51.887998 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 10:17:51.887998 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 10:17:51.888915 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 10:17:51.888915 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 10:17:51.888915 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 10:17:51.888915 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 10:17:51.888915 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:17:51.888915 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:17:51.888915 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:17:51.888915 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:17:51.888915 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:17:51.888915 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:17:51.888915 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:17:51.888915 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application with Lock( Jun 15 10:17:51.888915 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:17:51.888915 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:17:51.888915 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:17:51.888915 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:17:51.890280 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:17:51.890280 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:17:51.890280 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:17:51.890280 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:17:51.890280 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 10:17:51.890280 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 10:17:51.890280 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 10:17:51.890280 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 10:17:51.890280 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 10:17:51.890280 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 10:17:51.890280 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 10:17:51.890280 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application Jun 15 10:17:51.891380 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:17:51.893787 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 60/121] 10.222.0.49 () {56 vars in 1096 bytes} [Wed Jun 15 10:17:51 2022] GET /identity/v3/roles/admin => generated 83 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:17:51.900854 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51dad9c4-7d1a-4fc8-85e6-41cf816bf5fd None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:17:51.900986 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f3d8d0-8281-4f9a-8a8b-99e5f604a19f None admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:17:51.937174 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2bf55f-1f76-4f90-b465-edabd866662b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:17:51.938052 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2bf55f-1f76-4f90-b465-edabd866662b None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:17:51.938618 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2bf55f-1f76-4f90-b465-edabd866662b None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:17:51.938916 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2bf55f-1f76-4f90-b465-edabd866662b None admin] PATH_INFO: `/v3/projects/5f3cd857995e418386bb22cdf90c1ae9` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:17:51.940185 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2bf55f-1f76-4f90-b465-edabd866662b None admin] RBAC: Authorizing `identity:get_project(project_id=5f3cd857995e418386bb22cdf90c1ae9)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:17:51.941660 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2bf55f-1f76-4f90-b465-edabd866662b None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:17:51.948233 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 62/122] 10.222.0.49 () {56 vars in 1186 bytes} [Wed Jun 15 10:17:51 2022] GET /identity/v3/projects/5f3cd857995e418386bb22cdf90c1ae9 => generated 298 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:17:51.948233 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2bf55f-1f76-4f90-b465-edabd866662b None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:17:52.016840 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=100 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:17:52.026185 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f3d8d0-8281-4f9a-8a8b-99e5f604a19f None admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:17:52.063532 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2bf55f-1f76-4f90-b465-edabd866662b None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:17:52.073716 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef09b5f-3f3e-4711-8ec7-1f957395d8a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:17:52.074813 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef09b5f-3f3e-4711-8ec7-1f957395d8a4 None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:17:52.075415 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef09b5f-3f3e-4711-8ec7-1f957395d8a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:17:52.075914 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef09b5f-3f3e-4711-8ec7-1f957395d8a4 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:17:52.087875 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef09b5f-3f3e-4711-8ec7-1f957395d8a4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 10:17:52.097518 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application [None req-eef09b5f-3f3e-4711-8ec7-1f957395d8a4 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 15 10:17:52.097518 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:17:52.097518 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:17:52.097518 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:17:52.097518 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:17:52.097518 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:17:52.097518 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:17:52.097518 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:17:52.097518 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:17:52.097518 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:17:52.097518 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:17:52.097518 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:17:52.097518 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 10:17:52.097518 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 10:17:52.097518 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 10:17:52.097518 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 10:17:52.098502 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 10:17:52.098502 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 10:17:52.098502 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 10:17:52.098502 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 10:17:52.098502 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:17:52.098502 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:17:52.098502 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:17:52.098502 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:17:52.098502 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:17:52.098502 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:17:52.098502 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:17:52.098502 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application with Lock( Jun 15 10:17:52.098502 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:17:52.098502 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:17:52.098502 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:17:52.098502 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:17:52.099647 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:17:52.099647 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:17:52.099647 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:17:52.099647 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:17:52.099647 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 10:17:52.099647 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 10:17:52.099647 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 10:17:52.099647 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 10:17:52.099647 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 10:17:52.099647 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 10:17:52.099647 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 15 10:17:52.099647 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application Jun 15 10:17:52.102647 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 61/123] 10.222.0.49 () {56 vars in 1114 bytes} [Wed Jun 15 10:17:51 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:17:52.103709 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef09b5f-3f3e-4711-8ec7-1f957395d8a4 None admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:17:52.110914 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714cce2d-bc95-4559-838e-6ca6a8a033e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:17:52.111761 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714cce2d-bc95-4559-838e-6ca6a8a033e3 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:17:52.112058 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714cce2d-bc95-4559-838e-6ca6a8a033e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:17:52.112343 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714cce2d-bc95-4559-838e-6ca6a8a033e3 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:17:52.126169 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-714cce2d-bc95-4559-838e-6ca6a8a033e3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 10:17:52.136063 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application [None req-714cce2d-bc95-4559-838e-6ca6a8a033e3 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 15 10:17:52.136063 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:17:52.136063 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:17:52.136063 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:17:52.136063 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:17:52.136063 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:17:52.136063 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:17:52.136063 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:17:52.136063 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:17:52.136063 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:17:52.136063 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:17:52.136063 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:17:52.136063 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 10:17:52.136063 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 10:17:52.136063 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 10:17:52.136063 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 10:17:52.136968 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 10:17:52.136968 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 10:17:52.136968 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 10:17:52.136968 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 10:17:52.136968 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:17:52.136968 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:17:52.136968 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:17:52.136968 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:17:52.136968 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:17:52.136968 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:17:52.136968 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:17:52.136968 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application with Lock( Jun 15 10:17:52.136968 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:17:52.136968 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:17:52.136968 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:17:52.136968 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:17:52.137765 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:17:52.137765 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:17:52.137765 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:17:52.137765 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:17:52.137765 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 10:17:52.137765 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 10:17:52.137765 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 10:17:52.137765 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 10:17:52.137765 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 10:17:52.137765 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 10:17:52.137765 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 15 10:17:52.137765 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Jun 15 10:17:52.137765 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 63/124] 10.222.0.49 () {56 vars in 1099 bytes} [Wed Jun 15 10:17:51 2022] GET /identity/v3/roles/member => generated 84 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:17:52.138408 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714cce2d-bc95-4559-838e-6ca6a8a033e3 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:17:52.176120 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:17:52.236800 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef09b5f-3f3e-4711-8ec7-1f957395d8a4 None admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:17:52.260769 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714cce2d-bc95-4559-838e-6ca6a8a033e3 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:17:52.275053 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4309793-4adf-4482-82d6-00aa4e69ed9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:17:52.275774 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4309793-4adf-4482-82d6-00aa4e69ed9d None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:17:52.276067 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4309793-4adf-4482-82d6-00aa4e69ed9d None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:17:52.276354 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4309793-4adf-4482-82d6-00aa4e69ed9d None admin] PATH_INFO: `/v3/roles` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:17:52.277106 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4309793-4adf-4482-82d6-00aa4e69ed9d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:17:52.277420 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4309793-4adf-4482-82d6-00aa4e69ed9d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:17:52.278836 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4309793-4adf-4482-82d6-00aa4e69ed9d None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:17:52.288053 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 62/125] 10.222.0.49 () {56 vars in 1099 bytes} [Wed Jun 15 10:17:52 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:17:52.288913 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4309793-4adf-4482-82d6-00aa4e69ed9d None admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:17:52.294486 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf0b617-edf3-45a6-b240-8ad4849c1a2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:17:52.296042 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf0b617-edf3-45a6-b240-8ad4849c1a2b None admin] REQUEST_METHOD: `PUT` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:17:52.296328 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf0b617-edf3-45a6-b240-8ad4849c1a2b None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:17:52.296605 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf0b617-edf3-45a6-b240-8ad4849c1a2b None admin] PATH_INFO: `/v3/projects/5f3cd857995e418386bb22cdf90c1ae9/users/2f093b1bbc6148f39242d0692e1e8a95/roles/6876e57cc51c448097ab109d35b2a3da` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:17:52.298738 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf0b617-edf3-45a6-b240-8ad4849c1a2b None admin] RBAC: Authorizing `identity:create_grant(project_id=5f3cd857995e418386bb22cdf90c1ae9, user_id=2f093b1bbc6148f39242d0692e1e8a95, role_id=6876e57cc51c448097ab109d35b2a3da)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:17:52.300325 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf0b617-edf3-45a6-b240-8ad4849c1a2b None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:17:52.308207 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=100 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:17:52.316549 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 64/126] 10.222.0.49 () {58 vars in 1439 bytes} [Wed Jun 15 10:17:52 2022] PUT /identity/v3/projects/5f3cd857995e418386bb22cdf90c1ae9/users/2f093b1bbc6148f39242d0692e1e8a95/roles/6876e57cc51c448097ab109d35b2a3da => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:17:52.317932 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf0b617-edf3-45a6-b240-8ad4849c1a2b None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:17:52.407077 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4309793-4adf-4482-82d6-00aa4e69ed9d None admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:17:52.463490 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:17:52.481465 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf0b617-edf3-45a6-b240-8ad4849c1a2b None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:17:52.487899 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0b9c31-2c13-4c10-a889-2d2a9ef4bdb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:17:52.488617 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0b9c31-2c13-4c10-a889-2d2a9ef4bdb6 None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:17:52.488910 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0b9c31-2c13-4c10-a889-2d2a9ef4bdb6 None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:17:52.489193 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0b9c31-2c13-4c10-a889-2d2a9ef4bdb6 None admin] PATH_INFO: `/v3/roles` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:17:52.489993 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de0b9c31-2c13-4c10-a889-2d2a9ef4bdb6 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:17:52.490310 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de0b9c31-2c13-4c10-a889-2d2a9ef4bdb6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:17:52.495664 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de0b9c31-2c13-4c10-a889-2d2a9ef4bdb6 None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:17:52.513320 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 63/127] 10.222.0.49 () {56 vars in 1111 bytes} [Wed Jun 15 10:17:52 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:17:52.514262 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0b9c31-2c13-4c10-a889-2d2a9ef4bdb6 None admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:17:52.558599 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4914c691-5d49-45c1-9d66-c36f0b047203 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:17:52.559936 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4914c691-5d49-45c1-9d66-c36f0b047203 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:17:52.560367 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4914c691-5d49-45c1-9d66-c36f0b047203 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:17:52.560803 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4914c691-5d49-45c1-9d66-c36f0b047203 None admin] PATH_INFO: `/v3/roles` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:17:52.561781 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4914c691-5d49-45c1-9d66-c36f0b047203 None admin] RBAC: Adding query filter params (name=member) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:17:52.562849 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4914c691-5d49-45c1-9d66-c36f0b047203 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:17:52.564737 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4914c691-5d49-45c1-9d66-c36f0b047203 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:17:52.575947 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 65/128] 10.222.0.49 () {56 vars in 1101 bytes} [Wed Jun 15 10:17:52 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:17:52.578210 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4914c691-5d49-45c1-9d66-c36f0b047203 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:17:52.636351 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=100 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:17:52.663431 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0b9c31-2c13-4c10-a889-2d2a9ef4bdb6 None admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:17:52.709990 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4914c691-5d49-45c1-9d66-c36f0b047203 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:17:52.727044 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683739f8-2950-4ae3-9923-81777f156b7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:17:52.729337 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683739f8-2950-4ae3-9923-81777f156b7a None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:17:52.729631 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683739f8-2950-4ae3-9923-81777f156b7a None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:17:52.729914 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683739f8-2950-4ae3-9923-81777f156b7a None admin] PATH_INFO: `/v3/users/fcdb2d8e71bd4fe6b9b1431183facba7` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:17:52.732342 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-683739f8-2950-4ae3-9923-81777f156b7a None admin] RBAC: Authorizing `identity:get_user(user_id=fcdb2d8e71bd4fe6b9b1431183facba7)` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:17:52.736252 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-683739f8-2950-4ae3-9923-81777f156b7a None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:17:52.738758 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 64/129] 10.222.0.49 () {56 vars in 1177 bytes} [Wed Jun 15 10:17:52 2022] GET /identity/v3/users/fcdb2d8e71bd4fe6b9b1431183facba7 => generated 247 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:17:52.739561 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683739f8-2950-4ae3-9923-81777f156b7a None admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:17:52.780898 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ad53f3-c44e-4ff2-81f4-a69296c0e290 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:17:52.781935 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ad53f3-c44e-4ff2-81f4-a69296c0e290 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:17:52.782338 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ad53f3-c44e-4ff2-81f4-a69296c0e290 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:17:52.782718 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ad53f3-c44e-4ff2-81f4-a69296c0e290 None admin] PATH_INFO: `/v3/users/2f093b1bbc6148f39242d0692e1e8a95` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:17:52.787809 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ad53f3-c44e-4ff2-81f4-a69296c0e290 None admin] RBAC: Authorizing `identity:get_user(user_id=2f093b1bbc6148f39242d0692e1e8a95)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:17:52.790984 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ad53f3-c44e-4ff2-81f4-a69296c0e290 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:17:52.795424 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 66/130] 10.222.0.49 () {56 vars in 1177 bytes} [Wed Jun 15 10:17:52 2022] GET /identity/v3/users/2f093b1bbc6148f39242d0692e1e8a95 => generated 275 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:17:52.795424 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ad53f3-c44e-4ff2-81f4-a69296c0e290 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:17:52.841171 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:17:52.873366 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683739f8-2950-4ae3-9923-81777f156b7a None admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:17:52.909976 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db786e1b-c02b-432e-8bb2-d3bcc7a15379 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:17:52.911710 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db786e1b-c02b-432e-8bb2-d3bcc7a15379 None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:17:52.912233 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db786e1b-c02b-432e-8bb2-d3bcc7a15379 None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:17:52.912713 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db786e1b-c02b-432e-8bb2-d3bcc7a15379 None admin] PATH_INFO: `/v3/users/2f093b1bbc6148f39242d0692e1e8a95` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:17:52.914206 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db786e1b-c02b-432e-8bb2-d3bcc7a15379 None admin] RBAC: Authorizing `identity:get_user(user_id=2f093b1bbc6148f39242d0692e1e8a95)` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:17:52.915938 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db786e1b-c02b-432e-8bb2-d3bcc7a15379 None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:17:52.918102 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 65/131] 10.222.0.49 () {56 vars in 1177 bytes} [Wed Jun 15 10:17:52 2022] GET /identity/v3/users/2f093b1bbc6148f39242d0692e1e8a95 => generated 275 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:17:52.919161 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db786e1b-c02b-432e-8bb2-d3bcc7a15379 None admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:17:52.937393 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=100 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:17:53.029476 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db786e1b-c02b-432e-8bb2-d3bcc7a15379 None admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:17:53.077324 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999f480f-73d1-4694-9094-e50331a29f52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:17:53.079399 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999f480f-73d1-4694-9094-e50331a29f52 None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:17:53.081341 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999f480f-73d1-4694-9094-e50331a29f52 None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:17:53.082811 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999f480f-73d1-4694-9094-e50331a29f52 None admin] PATH_INFO: `/v3/projects/5f3cd857995e418386bb22cdf90c1ae9` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:17:53.085017 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-999f480f-73d1-4694-9094-e50331a29f52 None admin] RBAC: Authorizing `identity:get_project(project_id=5f3cd857995e418386bb22cdf90c1ae9)` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:17:53.088165 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-999f480f-73d1-4694-9094-e50331a29f52 None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:17:53.098394 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ad53f3-c44e-4ff2-81f4-a69296c0e290 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:17:53.099399 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 66/132] 10.222.0.49 () {56 vars in 1186 bytes} [Wed Jun 15 10:17:52 2022] GET /identity/v3/projects/5f3cd857995e418386bb22cdf90c1ae9 => generated 298 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:17:53.100170 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999f480f-73d1-4694-9094-e50331a29f52 None admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:17:53.138030 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89396970-f34f-4097-ad96-d8dc1851c939 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:17:53.139870 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89396970-f34f-4097-ad96-d8dc1851c939 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:17:53.140400 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89396970-f34f-4097-ad96-d8dc1851c939 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:17:53.140879 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89396970-f34f-4097-ad96-d8dc1851c939 None admin] PATH_INFO: `/v3/projects/5f3cd857995e418386bb22cdf90c1ae9` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:17:53.142673 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89396970-f34f-4097-ad96-d8dc1851c939 None admin] RBAC: Authorizing `identity:get_project(project_id=5f3cd857995e418386bb22cdf90c1ae9)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:17:53.144429 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89396970-f34f-4097-ad96-d8dc1851c939 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:17:53.146456 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 67/133] 10.222.0.49 () {56 vars in 1186 bytes} [Wed Jun 15 10:17:52 2022] GET /identity/v3/projects/5f3cd857995e418386bb22cdf90c1ae9 => generated 298 bytes in 355 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:17:53.147534 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89396970-f34f-4097-ad96-d8dc1851c939 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:17:53.257792 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89396970-f34f-4097-ad96-d8dc1851c939 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:17:53.287496 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:17:53.295651 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27cc39f-3984-4a64-a950-782c6ed69f17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:17:53.296930 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27cc39f-3984-4a64-a950-782c6ed69f17 None admin] REQUEST_METHOD: `PUT` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:17:53.297588 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27cc39f-3984-4a64-a950-782c6ed69f17 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:17:53.298119 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27cc39f-3984-4a64-a950-782c6ed69f17 None admin] PATH_INFO: `/v3/projects/5f3cd857995e418386bb22cdf90c1ae9/users/2f093b1bbc6148f39242d0692e1e8a95/roles/4d2118569d074a288b37d120110c300d` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:17:53.300989 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d27cc39f-3984-4a64-a950-782c6ed69f17 None admin] RBAC: Authorizing `identity:create_grant(project_id=5f3cd857995e418386bb22cdf90c1ae9, user_id=2f093b1bbc6148f39242d0692e1e8a95, role_id=4d2118569d074a288b37d120110c300d)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:17:53.302670 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d27cc39f-3984-4a64-a950-782c6ed69f17 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:17:53.321245 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 68/134] 10.222.0.49 () {58 vars in 1439 bytes} [Wed Jun 15 10:17:53 2022] PUT /identity/v3/projects/5f3cd857995e418386bb22cdf90c1ae9/users/2f093b1bbc6148f39242d0692e1e8a95/roles/4d2118569d074a288b37d120110c300d => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:17:53.322378 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27cc39f-3984-4a64-a950-782c6ed69f17 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:17:53.419872 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=100 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:17:53.477367 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999f480f-73d1-4694-9094-e50331a29f52 None admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:17:53.560058 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4584ad-7598-46e9-b068-e05868407e06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:17:53.561901 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4584ad-7598-46e9-b068-e05868407e06 None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:17:53.562836 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4584ad-7598-46e9-b068-e05868407e06 None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:17:53.563366 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4584ad-7598-46e9-b068-e05868407e06 None admin] PATH_INFO: `/v3/projects/24ea2a1e0d1f49bebb8a85e96ebbe085` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:17:53.565276 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a4584ad-7598-46e9-b068-e05868407e06 None admin] RBAC: Authorizing `identity:get_project(project_id=24ea2a1e0d1f49bebb8a85e96ebbe085)` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:17:53.566972 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a4584ad-7598-46e9-b068-e05868407e06 None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:17:53.569383 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 67/135] 10.222.0.49 () {56 vars in 1186 bytes} [Wed Jun 15 10:17:53 2022] GET /identity/v3/projects/24ea2a1e0d1f49bebb8a85e96ebbe085 => generated 312 bytes in 469 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:17:53.574747 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4584ad-7598-46e9-b068-e05868407e06 None admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:17:53.638493 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27cc39f-3984-4a64-a950-782c6ed69f17 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:17:53.696708 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e875fcd3-b5b2-4613-aa07-b6e7bbba51ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:17:53.697618 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e875fcd3-b5b2-4613-aa07-b6e7bbba51ce None admin] REQUEST_METHOD: `PUT` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:17:53.697908 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e875fcd3-b5b2-4613-aa07-b6e7bbba51ce None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:17:53.698189 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e875fcd3-b5b2-4613-aa07-b6e7bbba51ce None admin] PATH_INFO: `/v3/projects/5f3cd857995e418386bb22cdf90c1ae9/users/fcdb2d8e71bd4fe6b9b1431183facba7/roles/8051174e7a7947d5b991ebd793c2e543` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:17:53.700371 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e875fcd3-b5b2-4613-aa07-b6e7bbba51ce None admin] RBAC: Authorizing `identity:create_grant(project_id=5f3cd857995e418386bb22cdf90c1ae9, user_id=fcdb2d8e71bd4fe6b9b1431183facba7, role_id=8051174e7a7947d5b991ebd793c2e543)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:17:53.701873 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e875fcd3-b5b2-4613-aa07-b6e7bbba51ce None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:17:53.717228 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 69/136] 10.222.0.49 () {58 vars in 1439 bytes} [Wed Jun 15 10:17:53 2022] PUT /identity/v3/projects/5f3cd857995e418386bb22cdf90c1ae9/users/fcdb2d8e71bd4fe6b9b1431183facba7/roles/8051174e7a7947d5b991ebd793c2e543 => generated 0 bytes in 395 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:17:53.718521 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5330973-66fe-4ed9-a36e-dd89edc62ed2 None None] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:17:53.718817 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5330973-66fe-4ed9-a36e-dd89edc62ed2 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:17:53.719113 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5330973-66fe-4ed9-a36e-dd89edc62ed2 None None] PATH_INFO: `/` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:17:53.723957 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4584ad-7598-46e9-b068-e05868407e06 None admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:17:53.724615 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 70/137] 10.222.0.49 () {54 vars in 926 bytes} [Wed Jun 15 10:17:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:17:53.727538 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f926bda4-288d-45e7-a5d2-f1d6fbe21554 None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:17:53.727538 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f926bda4-288d-45e7-a5d2-f1d6fbe21554 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:17:53.727729 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f926bda4-288d-45e7-a5d2-f1d6fbe21554 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:17:53.745596 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:17:53.756856 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-f926bda4-288d-45e7-a5d2-f1d6fbe21554 None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:17:53.760725 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-f926bda4-288d-45e7-a5d2-f1d6fbe21554 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:17:53.775815 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b20488-9f8c-4b16-93f4-d36ccd3d662d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:17:53.776771 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b20488-9f8c-4b16-93f4-d36ccd3d662d None admin] REQUEST_METHOD: `PUT` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:17:53.777065 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b20488-9f8c-4b16-93f4-d36ccd3d662d None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:17:53.777355 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b20488-9f8c-4b16-93f4-d36ccd3d662d None admin] PATH_INFO: `/v3/projects/24ea2a1e0d1f49bebb8a85e96ebbe085/users/2f093b1bbc6148f39242d0692e1e8a95/roles/6876e57cc51c448097ab109d35b2a3da` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:17:53.780259 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b20488-9f8c-4b16-93f4-d36ccd3d662d None admin] RBAC: Authorizing `identity:create_grant(project_id=24ea2a1e0d1f49bebb8a85e96ebbe085, user_id=2f093b1bbc6148f39242d0692e1e8a95, role_id=6876e57cc51c448097ab109d35b2a3da)` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:17:53.782793 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b20488-9f8c-4b16-93f4-d36ccd3d662d None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:17:53.815788 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 68/138] 10.222.0.49 () {58 vars in 1439 bytes} [Wed Jun 15 10:17:53 2022] PUT /identity/v3/projects/24ea2a1e0d1f49bebb8a85e96ebbe085/users/2f093b1bbc6148f39242d0692e1e8a95/roles/6876e57cc51c448097ab109d35b2a3da => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:17:53.912914 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 71/139] 10.222.0.49 () {58 vars in 1023 bytes} [Wed Jun 15 10:17:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 15 10:17:53.956996 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b20488-9f8c-4b16-93f4-d36ccd3d662d None admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:17:53.977089 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:17:54.086579 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b20488-9f8c-4b16-93f4-d36ccd3d662d None admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:17:54.126921 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840398e3-23d9-448c-a763-a35b74a2edb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:17:54.127762 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840398e3-23d9-448c-a763-a35b74a2edb5 None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:17:54.128056 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840398e3-23d9-448c-a763-a35b74a2edb5 None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:17:54.128341 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840398e3-23d9-448c-a763-a35b74a2edb5 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:17:54.129554 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-840398e3-23d9-448c-a763-a35b74a2edb5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:17:54.131048 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-840398e3-23d9-448c-a763-a35b74a2edb5 None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:17:54.133020 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 69/140] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Jun 15 10:17:53 2022] GET /identity/v3/domains/default => generated 199 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:17:54.138301 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f926bda4-288d-45e7-a5d2-f1d6fbe21554 None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:17:54.270281 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f926bda4-288d-45e7-a5d2-f1d6fbe21554 None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:17:54.320606 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d12845-d1cf-4d48-ba1f-3efefcca7d63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:17:54.321685 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d12845-d1cf-4d48-ba1f-3efefcca7d63 None admin] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:17:54.322166 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d12845-d1cf-4d48-ba1f-3efefcca7d63 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:17:54.322662 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d12845-d1cf-4d48-ba1f-3efefcca7d63 None admin] PATH_INFO: `/v3/users` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:17:54.323632 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d12845-d1cf-4d48-ba1f-3efefcca7d63 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:17:54.325356 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d12845-d1cf-4d48-ba1f-3efefcca7d63 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:17:54.337562 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:17:54.359415 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 72/141] 10.222.0.49 () {60 vars in 1132 bytes} [Wed Jun 15 10:17:54 2022] POST /identity/v3/users => generated 283 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 10:17:54.426338 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee93386-1aab-43af-a042-e734fbdcc403 None None] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:17:54.426338 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee93386-1aab-43af-a042-e734fbdcc403 None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:17:54.426338 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee93386-1aab-43af-a042-e734fbdcc403 None None] PATH_INFO: `/` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:17:54.426338 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 70/142] 10.222.0.49 () {54 vars in 926 bytes} [Wed Jun 15 10:17:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:17:54.432996 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a38ab52-e05a-4064-ba49-c5498ae6d569 None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:17:54.433295 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a38ab52-e05a-4064-ba49-c5498ae6d569 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:17:54.433581 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a38ab52-e05a-4064-ba49-c5498ae6d569 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:17:54.456336 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-2a38ab52-e05a-4064-ba49-c5498ae6d569 None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:17:54.457579 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-2a38ab52-e05a-4064-ba49-c5498ae6d569 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:17:54.544560 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 73/143] 10.222.0.49 () {58 vars in 1023 bytes} [Wed Jun 15 10:17:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 10:17:54.574651 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee93386-1aab-43af-a042-e734fbdcc403 None None] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:17:54.695037 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=100 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:17:54.719749 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee93386-1aab-43af-a042-e734fbdcc403 None None] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:17:54.757488 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13412b9e-b574-443e-9425-15ce3df5b078 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:17:54.758298 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13412b9e-b574-443e-9425-15ce3df5b078 None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:17:54.758580 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13412b9e-b574-443e-9425-15ce3df5b078 None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:17:54.758858 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13412b9e-b574-443e-9425-15ce3df5b078 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:17:54.771667 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13412b9e-b574-443e-9425-15ce3df5b078 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 10:17:54.778717 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application [None req-13412b9e-b574-443e-9425-15ce3df5b078 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 15 10:17:54.778717 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:17:54.778717 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:17:54.778717 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:17:54.778717 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:17:54.778717 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:17:54.778717 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:17:54.778717 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:17:54.778717 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:17:54.778717 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:17:54.778717 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:17:54.778717 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:17:54.778717 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 10:17:54.778717 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 10:17:54.778717 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 10:17:54.778717 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 10:17:54.779678 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 10:17:54.779678 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 10:17:54.779678 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 10:17:54.779678 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 10:17:54.779678 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:17:54.779678 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:17:54.779678 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:17:54.779678 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:17:54.779678 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:17:54.779678 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:17:54.779678 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:17:54.779678 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application with Lock( Jun 15 10:17:54.779678 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:17:54.779678 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:17:54.779678 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:17:54.779678 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:17:54.780458 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:17:54.780458 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:17:54.780458 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:17:54.780458 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:17:54.780458 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 10:17:54.780458 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 10:17:54.780458 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 10:17:54.780458 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 10:17:54.780458 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 10:17:54.780458 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 10:17:54.780458 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 15 10:17:54.780458 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application Jun 15 10:17:54.780458 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 71/144] 10.222.0.49 () {56 vars in 1099 bytes} [Wed Jun 15 10:17:54 2022] GET /identity/v3/roles/reader => generated 84 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:17:54.787357 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a38ab52-e05a-4064-ba49-c5498ae6d569 None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:17:54.892721 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a38ab52-e05a-4064-ba49-c5498ae6d569 None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:17:54.913663 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:17:54.930169 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52a8644-1b89-4e9b-a293-8e99822d1386 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:17:54.930877 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52a8644-1b89-4e9b-a293-8e99822d1386 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:17:54.931187 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52a8644-1b89-4e9b-a293-8e99822d1386 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:17:54.931464 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52a8644-1b89-4e9b-a293-8e99822d1386 None admin] PATH_INFO: `/v3/roles` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:17:54.932258 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52a8644-1b89-4e9b-a293-8e99822d1386 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:17:54.932561 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52a8644-1b89-4e9b-a293-8e99822d1386 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:17:54.933936 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52a8644-1b89-4e9b-a293-8e99822d1386 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:17:54.943174 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 74/145] 10.222.0.49 () {56 vars in 1101 bytes} [Wed Jun 15 10:17:54 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:17:54.948755 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13412b9e-b574-443e-9425-15ce3df5b078 None admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:17:55.070751 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13412b9e-b574-443e-9425-15ce3df5b078 None admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:17:55.109247 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f6cefe-9cfe-48ce-813c-bfa69b58c581 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:17:55.110145 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f6cefe-9cfe-48ce-813c-bfa69b58c581 None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:17:55.110430 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f6cefe-9cfe-48ce-813c-bfa69b58c581 None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:17:55.110708 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f6cefe-9cfe-48ce-813c-bfa69b58c581 None admin] PATH_INFO: `/v3/users/dfe1d4745a4d4642a21f8860b4a02fb2` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:17:55.111972 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f6cefe-9cfe-48ce-813c-bfa69b58c581 None admin] RBAC: Authorizing `identity:get_user(user_id=dfe1d4745a4d4642a21f8860b4a02fb2)` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:17:55.113439 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f6cefe-9cfe-48ce-813c-bfa69b58c581 None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:17:55.119509 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 72/146] 10.222.0.49 () {56 vars in 1177 bytes} [Wed Jun 15 10:17:54 2022] GET /identity/v3/users/dfe1d4745a4d4642a21f8860b4a02fb2 => generated 289 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:17:55.120860 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52a8644-1b89-4e9b-a293-8e99822d1386 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:17:55.225196 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52a8644-1b89-4e9b-a293-8e99822d1386 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:17:55.260175 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08af4a7e-d5c4-452d-9f80-b3b06df6f980 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:17:55.261049 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08af4a7e-d5c4-452d-9f80-b3b06df6f980 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:17:55.261332 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08af4a7e-d5c4-452d-9f80-b3b06df6f980 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:17:55.261606 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08af4a7e-d5c4-452d-9f80-b3b06df6f980 None admin] PATH_INFO: `/v3/projects/5f3cd857995e418386bb22cdf90c1ae9` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:17:55.262853 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08af4a7e-d5c4-452d-9f80-b3b06df6f980 None admin] RBAC: Authorizing `identity:get_project(project_id=5f3cd857995e418386bb22cdf90c1ae9)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:17:55.264421 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08af4a7e-d5c4-452d-9f80-b3b06df6f980 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:17:55.266210 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 75/147] 10.222.0.49 () {56 vars in 1186 bytes} [Wed Jun 15 10:17:55 2022] GET /identity/v3/projects/5f3cd857995e418386bb22cdf90c1ae9 => generated 298 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:17:55.272017 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f6cefe-9cfe-48ce-813c-bfa69b58c581 None admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:17:55.283951 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=100 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:17:55.384710 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aba5048-14ba-4aaa-9ecd-765116242a81 None None] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:17:55.385033 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aba5048-14ba-4aaa-9ecd-765116242a81 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:17:55.385256 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aba5048-14ba-4aaa-9ecd-765116242a81 None None] PATH_INFO: `/` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:17:55.386629 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 76/148] 10.222.0.49 () {54 vars in 926 bytes} [Wed Jun 15 10:17:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 15 10:17:55.392797 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306c5962-7a5b-4a2c-884c-4f32a71adcba None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:17:55.393077 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306c5962-7a5b-4a2c-884c-4f32a71adcba None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:17:55.393351 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306c5962-7a5b-4a2c-884c-4f32a71adcba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:17:55.396817 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f6cefe-9cfe-48ce-813c-bfa69b58c581 None admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:17:55.416159 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-306c5962-7a5b-4a2c-884c-4f32a71adcba None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:17:55.417721 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-306c5962-7a5b-4a2c-884c-4f32a71adcba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:17:55.455022 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211a24d9-7f8c-496d-a0b9-790090814dab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:17:55.456105 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211a24d9-7f8c-496d-a0b9-790090814dab None admin] REQUEST_METHOD: `PUT` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:17:55.456407 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211a24d9-7f8c-496d-a0b9-790090814dab None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:17:55.456873 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211a24d9-7f8c-496d-a0b9-790090814dab None admin] PATH_INFO: `/v3/projects/5f3cd857995e418386bb22cdf90c1ae9/users/dfe1d4745a4d4642a21f8860b4a02fb2/roles/301a3c8a23ba40edbc35df1da8af0de9` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:17:55.459461 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-211a24d9-7f8c-496d-a0b9-790090814dab None admin] RBAC: Authorizing `identity:create_grant(project_id=5f3cd857995e418386bb22cdf90c1ae9, user_id=dfe1d4745a4d4642a21f8860b4a02fb2, role_id=301a3c8a23ba40edbc35df1da8af0de9)` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:17:55.461315 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-211a24d9-7f8c-496d-a0b9-790090814dab None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:17:55.480027 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 73/149] 10.222.0.49 () {58 vars in 1439 bytes} [Wed Jun 15 10:17:55 2022] PUT /identity/v3/projects/5f3cd857995e418386bb22cdf90c1ae9/users/dfe1d4745a4d4642a21f8860b4a02fb2/roles/301a3c8a23ba40edbc35df1da8af0de9 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:17:55.516267 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:17:55.546191 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 77/150] 10.222.0.49 () {58 vars in 1023 bytes} [Wed Jun 15 10:17:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 15 10:17:55.588269 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211a24d9-7f8c-496d-a0b9-790090814dab None admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:17:55.673431 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:17:55.716284 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211a24d9-7f8c-496d-a0b9-790090814dab None admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:17:55.757533 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623beb45-3c58-4c34-b91f-96423952de20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:17:55.758305 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623beb45-3c58-4c34-b91f-96423952de20 None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:17:55.758592 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623beb45-3c58-4c34-b91f-96423952de20 None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:17:55.758870 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623beb45-3c58-4c34-b91f-96423952de20 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:17:55.775123 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-623beb45-3c58-4c34-b91f-96423952de20 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 10:17:55.782290 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application [None req-623beb45-3c58-4c34-b91f-96423952de20 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 15 10:17:55.782290 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:17:55.782290 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:17:55.782290 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:17:55.782290 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:17:55.782290 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:17:55.782290 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:17:55.782290 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:17:55.782290 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:17:55.782290 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:17:55.782290 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:17:55.782290 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:17:55.782290 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 10:17:55.782290 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 10:17:55.782290 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 10:17:55.782290 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 10:17:55.783206 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 10:17:55.783206 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 10:17:55.783206 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 10:17:55.783206 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 10:17:55.783206 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:17:55.783206 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:17:55.783206 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:17:55.783206 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:17:55.783206 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:17:55.783206 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:17:55.783206 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:17:55.783206 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application with Lock( Jun 15 10:17:55.783206 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:17:55.783206 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:17:55.783206 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:17:55.783206 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:17:55.784084 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:17:55.784084 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:17:55.784084 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:17:55.784084 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:17:55.784084 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 10:17:55.784084 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 10:17:55.784084 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 10:17:55.784084 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 10:17:55.784084 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 10:17:55.784084 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 10:17:55.784084 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 15 10:17:55.784084 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application Jun 15 10:17:55.784084 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 74/151] 10.222.0.49 () {56 vars in 1099 bytes} [Wed Jun 15 10:17:55 2022] GET /identity/v3/roles/member => generated 84 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:17:55.794182 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306c5962-7a5b-4a2c-884c-4f32a71adcba None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:17:55.923958 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306c5962-7a5b-4a2c-884c-4f32a71adcba None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:17:55.964735 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8a7978-4369-4180-af1b-52af7e87d9cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:17:55.965442 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8a7978-4369-4180-af1b-52af7e87d9cd None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:17:55.965733 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8a7978-4369-4180-af1b-52af7e87d9cd None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:17:55.966014 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8a7978-4369-4180-af1b-52af7e87d9cd None admin] PATH_INFO: `/v3/roles` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:17:55.966775 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af8a7978-4369-4180-af1b-52af7e87d9cd None admin] RBAC: Adding query filter params (name=member) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:17:55.967740 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af8a7978-4369-4180-af1b-52af7e87d9cd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:17:55.969148 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af8a7978-4369-4180-af1b-52af7e87d9cd None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:17:55.978126 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 78/152] 10.222.0.49 () {56 vars in 1101 bytes} [Wed Jun 15 10:17:55 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:17:55.983701 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623beb45-3c58-4c34-b91f-96423952de20 None admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:17:56.120035 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623beb45-3c58-4c34-b91f-96423952de20 None admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:17:56.159500 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=100 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:17:56.176039 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1bc72ca-34a1-41de-b6db-fa698154e701 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:17:56.177293 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bc72ca-34a1-41de-b6db-fa698154e701 None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:17:56.178038 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bc72ca-34a1-41de-b6db-fa698154e701 None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:17:56.178536 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bc72ca-34a1-41de-b6db-fa698154e701 None admin] PATH_INFO: `/v3/users/2f093b1bbc6148f39242d0692e1e8a95` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:17:56.187201 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1bc72ca-34a1-41de-b6db-fa698154e701 None admin] RBAC: Authorizing `identity:get_user(user_id=2f093b1bbc6148f39242d0692e1e8a95)` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:17:56.188382 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1bc72ca-34a1-41de-b6db-fa698154e701 None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:17:56.191774 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 75/153] 10.222.0.49 () {56 vars in 1177 bytes} [Wed Jun 15 10:17:55 2022] GET /identity/v3/users/2f093b1bbc6148f39242d0692e1e8a95 => generated 275 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:17:56.212257 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8a7978-4369-4180-af1b-52af7e87d9cd None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:17:56.338023 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:17:56.473399 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8a7978-4369-4180-af1b-52af7e87d9cd None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:17:56.559821 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0791da87-d18e-4cbd-ba1e-932c043050b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:17:56.561473 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0791da87-d18e-4cbd-ba1e-932c043050b1 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:17:56.561991 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0791da87-d18e-4cbd-ba1e-932c043050b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:17:56.562472 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0791da87-d18e-4cbd-ba1e-932c043050b1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:17:56.714003 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0791da87-d18e-4cbd-ba1e-932c043050b1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:17:56.715881 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0791da87-d18e-4cbd-ba1e-932c043050b1 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:17:56.777710 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:17:56.850692 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 79/154] 10.222.0.49 () {58 vars in 1282 bytes} [Wed Jun 15 10:17:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 639 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 15 10:17:56.856199 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1bc72ca-34a1-41de-b6db-fa698154e701 None admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:17:56.964887 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1bc72ca-34a1-41de-b6db-fa698154e701 None admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:17:57.010507 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9397ec4c-29d8-4c2b-a62f-caf0bf10329a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:17:57.011412 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9397ec4c-29d8-4c2b-a62f-caf0bf10329a None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:17:57.011695 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9397ec4c-29d8-4c2b-a62f-caf0bf10329a None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:17:57.011974 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9397ec4c-29d8-4c2b-a62f-caf0bf10329a None admin] PATH_INFO: `/v3/projects/5f3cd857995e418386bb22cdf90c1ae9` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:17:57.013174 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9397ec4c-29d8-4c2b-a62f-caf0bf10329a None admin] RBAC: Authorizing `identity:get_project(project_id=5f3cd857995e418386bb22cdf90c1ae9)` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:17:57.015399 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9397ec4c-29d8-4c2b-a62f-caf0bf10329a None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:17:57.017103 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 76/155] 10.222.0.49 () {56 vars in 1186 bytes} [Wed Jun 15 10:17:56 2022] GET /identity/v3/projects/5f3cd857995e418386bb22cdf90c1ae9 => generated 298 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 10:17:57.023208 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0791da87-d18e-4cbd-ba1e-932c043050b1 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:17:57.147750 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0791da87-d18e-4cbd-ba1e-932c043050b1 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:17:57.190025 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf5628e-365c-4d07-8636-f6571783ff1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:17:57.190695 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf5628e-365c-4d07-8636-f6571783ff1a None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:17:57.190976 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf5628e-365c-4d07-8636-f6571783ff1a None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:17:57.191275 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf5628e-365c-4d07-8636-f6571783ff1a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:17:57.192126 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf5628e-365c-4d07-8636-f6571783ff1a None admin] RBAC: Adding query filter params (role.id=6876e57cc51c448097ab109d35b2a3da, scope.project.id=5f3cd857995e418386bb22cdf90c1ae9, user.id=2f093b1bbc6148f39242d0692e1e8a95) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:17:57.192438 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf5628e-365c-4d07-8636-f6571783ff1a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:17:57.193961 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf5628e-365c-4d07-8636-f6571783ff1a None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:17:57.204347 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 80/156] 10.222.0.49 () {56 vars in 1374 bytes} [Wed Jun 15 10:17:57 2022] GET /identity/v3/role_assignments?user.id=2f093b1bbc6148f39242d0692e1e8a95&scope.project.id=5f3cd857995e418386bb22cdf90c1ae9&role.id=6876e57cc51c448097ab109d35b2a3da => generated 608 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:17:57.366966 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489686b7-0ccd-43c4-96b2-4cbcebf9e1d9 None None] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:17:57.367497 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489686b7-0ccd-43c4-96b2-4cbcebf9e1d9 None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:17:57.367594 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489686b7-0ccd-43c4-96b2-4cbcebf9e1d9 None None] PATH_INFO: `/` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:17:57.369094 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 77/157] 10.222.0.49 () {54 vars in 926 bytes} [Wed Jun 15 10:17:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:17:57.375650 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c73c230-24ac-41ae-9fbb-ec929a69da1e None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:17:57.376098 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c73c230-24ac-41ae-9fbb-ec929a69da1e None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:17:57.376283 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c73c230-24ac-41ae-9fbb-ec929a69da1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:17:57.388143 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:17:57.394585 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-1c73c230-24ac-41ae-9fbb-ec929a69da1e None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:17:57.395839 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-1c73c230-24ac-41ae-9fbb-ec929a69da1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:17:57.478329 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 81/158] 10.222.0.49 () {58 vars in 1023 bytes} [Wed Jun 15 10:17:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 10:17:57.534428 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489686b7-0ccd-43c4-96b2-4cbcebf9e1d9 None None] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:17:57.611297 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=100 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:17:57.660904 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489686b7-0ccd-43c4-96b2-4cbcebf9e1d9 None None] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:17:57.702674 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91737db-1f7e-418f-a5f6-ce526199d0c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:17:57.703574 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91737db-1f7e-418f-a5f6-ce526199d0c9 None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:17:57.703858 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91737db-1f7e-418f-a5f6-ce526199d0c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:17:57.704135 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91737db-1f7e-418f-a5f6-ce526199d0c9 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:17:57.717608 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a91737db-1f7e-418f-a5f6-ce526199d0c9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 10:17:57.725896 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application [None req-a91737db-1f7e-418f-a5f6-ce526199d0c9 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 15 10:17:57.725896 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:17:57.725896 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:17:57.725896 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:17:57.725896 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:17:57.725896 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:17:57.725896 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:17:57.725896 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:17:57.725896 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:17:57.725896 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:17:57.725896 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:17:57.725896 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:17:57.725896 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 10:17:57.725896 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 10:17:57.725896 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 10:17:57.725896 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 10:17:57.726757 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 10:17:57.726757 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 10:17:57.726757 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 10:17:57.726757 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 10:17:57.726757 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:17:57.726757 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:17:57.726757 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:17:57.726757 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:17:57.726757 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:17:57.726757 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:17:57.726757 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:17:57.726757 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application with Lock( Jun 15 10:17:57.726757 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:17:57.726757 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:17:57.726757 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:17:57.726757 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:17:57.728035 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:17:57.728035 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:17:57.728035 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:17:57.728035 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:17:57.728035 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 10:17:57.728035 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 10:17:57.728035 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 10:17:57.728035 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 10:17:57.728035 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 10:17:57.728035 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 10:17:57.728035 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 15 10:17:57.728035 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application Jun 15 10:17:57.728035 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 78/159] 10.222.0.49 () {56 vars in 1114 bytes} [Wed Jun 15 10:17:57 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:17:57.735375 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c73c230-24ac-41ae-9fbb-ec929a69da1e None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:17:57.876843 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ee4cae-c686-47f1-94b8-18687190ee63 None None] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:17:57.877153 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ee4cae-c686-47f1-94b8-18687190ee63 None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:17:57.877399 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ee4cae-c686-47f1-94b8-18687190ee63 None None] PATH_INFO: `/` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:17:57.878768 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 79/160] 10.222.0.49 () {54 vars in 926 bytes} [Wed Jun 15 10:17:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 15 10:17:57.883750 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d187b1-c6c3-4b1d-bff7-c95149719188 None None] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:17:57.884036 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d187b1-c6c3-4b1d-bff7-c95149719188 None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:17:57.884313 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d187b1-c6c3-4b1d-bff7-c95149719188 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:17:57.916114 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.auth.core [None req-77d187b1-c6c3-4b1d-bff7-c95149719188 None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:17:57.916547 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c73c230-24ac-41ae-9fbb-ec929a69da1e None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:17:57.917872 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-77d187b1-c6c3-4b1d-bff7-c95149719188 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:17:57.970272 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449b7a26-af27-4c40-9f8c-d44ea8b3de26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:17:57.970976 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449b7a26-af27-4c40-9f8c-d44ea8b3de26 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:17:57.971290 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449b7a26-af27-4c40-9f8c-d44ea8b3de26 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:17:57.971580 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449b7a26-af27-4c40-9f8c-d44ea8b3de26 None admin] PATH_INFO: `/v3/roles` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:17:57.972572 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-449b7a26-af27-4c40-9f8c-d44ea8b3de26 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:17:57.972926 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-449b7a26-af27-4c40-9f8c-d44ea8b3de26 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:17:57.974806 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-449b7a26-af27-4c40-9f8c-d44ea8b3de26 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:17:57.979204 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:17:57.989942 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 82/161] 10.222.0.49 () {56 vars in 1111 bytes} [Wed Jun 15 10:17:57 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:17:57.991502 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d18184d-a770-4a21-ab4a-d87f804edd3b None None] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:17:57.991931 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d18184d-a770-4a21-ab4a-d87f804edd3b None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:17:57.992807 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d18184d-a770-4a21-ab4a-d87f804edd3b None None] PATH_INFO: `/` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:17:57.994583 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 83/162] 10.222.0.49 () {54 vars in 926 bytes} [Wed Jun 15 10:17:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:17:58.000390 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6508504-15f8-40d8-a3dd-34e2f35cf89b None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:17:58.001099 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6508504-15f8-40d8-a3dd-34e2f35cf89b None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:17:58.001514 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6508504-15f8-40d8-a3dd-34e2f35cf89b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:17:58.028721 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-c6508504-15f8-40d8-a3dd-34e2f35cf89b None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:17:58.030518 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-c6508504-15f8-40d8-a3dd-34e2f35cf89b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:17:58.035233 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 80/163] 10.222.0.49 () {58 vars in 1023 bytes} [Wed Jun 15 10:17:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 15 10:17:58.036156 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d187b1-c6c3-4b1d-bff7-c95149719188 None None] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:17:58.133912 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 84/164] 10.222.0.49 () {58 vars in 1023 bytes} [Wed Jun 15 10:17:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 10:17:58.135922 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=100 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:17:58.136514 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6508504-15f8-40d8-a3dd-34e2f35cf89b None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:17:58.245203 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d187b1-c6c3-4b1d-bff7-c95149719188 None None] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:17:58.314697 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f752347-215b-49c8-8ed5-07a5395a227a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:17:58.317188 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6508504-15f8-40d8-a3dd-34e2f35cf89b None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:17:58.319341 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f752347-215b-49c8-8ed5-07a5395a227a None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:17:58.319631 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f752347-215b-49c8-8ed5-07a5395a227a None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:17:58.319912 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f752347-215b-49c8-8ed5-07a5395a227a None admin] PATH_INFO: `/v3/users/2f093b1bbc6148f39242d0692e1e8a95` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:17:58.321826 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f752347-215b-49c8-8ed5-07a5395a227a None admin] RBAC: Authorizing `identity:get_user(user_id=2f093b1bbc6148f39242d0692e1e8a95)` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:17:58.323756 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f752347-215b-49c8-8ed5-07a5395a227a None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:17:58.325445 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 81/165] 10.222.0.49 () {56 vars in 1177 bytes} [Wed Jun 15 10:17:58 2022] GET /identity/v3/users/2f093b1bbc6148f39242d0692e1e8a95 => generated 275 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:17:58.327896 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f752347-215b-49c8-8ed5-07a5395a227a None admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:17:58.375933 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:17:58.387053 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278bfd71-4fd3-4139-ac38-9f9e54771926 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:17:58.387888 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278bfd71-4fd3-4139-ac38-9f9e54771926 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:17:58.388178 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278bfd71-4fd3-4139-ac38-9f9e54771926 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:17:58.388459 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278bfd71-4fd3-4139-ac38-9f9e54771926 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:17:58.402592 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-278bfd71-4fd3-4139-ac38-9f9e54771926 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 10:17:58.418969 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application [None req-278bfd71-4fd3-4139-ac38-9f9e54771926 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 10:17:58.418969 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:17:58.418969 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:17:58.418969 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:17:58.418969 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:17:58.418969 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:17:58.418969 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:17:58.418969 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:17:58.418969 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:17:58.418969 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:17:58.418969 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:17:58.418969 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:17:58.418969 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 10:17:58.418969 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 10:17:58.418969 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 10:17:58.418969 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 10:17:58.419895 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 10:17:58.419895 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 10:17:58.419895 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 10:17:58.419895 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 10:17:58.419895 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:17:58.419895 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:17:58.419895 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:17:58.419895 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:17:58.419895 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:17:58.419895 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:17:58.419895 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:17:58.419895 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application with Lock( Jun 15 10:17:58.419895 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:17:58.419895 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:17:58.419895 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:17:58.419895 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:17:58.420795 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:17:58.420795 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:17:58.420795 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:17:58.420795 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:17:58.420795 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 10:17:58.420795 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 10:17:58.420795 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 10:17:58.420795 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 10:17:58.420795 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 10:17:58.420795 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 10:17:58.420795 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 10:17:58.420795 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Jun 15 10:17:58.420795 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 85/166] 10.222.0.49 () {56 vars in 1096 bytes} [Wed Jun 15 10:17:58 2022] GET /identity/v3/roles/admin => generated 83 bytes in 286 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:17:58.421679 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278bfd71-4fd3-4139-ac38-9f9e54771926 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:17:58.467042 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f752347-215b-49c8-8ed5-07a5395a227a None admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:17:58.488293 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=100 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:17:58.510528 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a438d32c-62d2-499d-ac16-0af7cc5c8e1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:17:58.512153 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a438d32c-62d2-499d-ac16-0af7cc5c8e1e None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:17:58.512449 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a438d32c-62d2-499d-ac16-0af7cc5c8e1e None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:17:58.512731 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a438d32c-62d2-499d-ac16-0af7cc5c8e1e None admin] PATH_INFO: `/v3/roles/member` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:17:58.535972 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a438d32c-62d2-499d-ac16-0af7cc5c8e1e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 10:17:58.550170 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application [None req-a438d32c-62d2-499d-ac16-0af7cc5c8e1e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 15 10:17:58.550170 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:17:58.550170 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:17:58.550170 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:17:58.550170 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:17:58.550170 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:17:58.550170 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:17:58.550170 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:17:58.550170 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:17:58.550170 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:17:58.550170 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:17:58.550170 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:17:58.550170 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 10:17:58.550170 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 10:17:58.550170 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 10:17:58.550170 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 10:17:58.551062 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 10:17:58.551062 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 10:17:58.551062 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 10:17:58.551062 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 10:17:58.551062 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:17:58.551062 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:17:58.551062 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:17:58.551062 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:17:58.551062 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:17:58.551062 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:17:58.551062 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:17:58.551062 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application with Lock( Jun 15 10:17:58.551062 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:17:58.551062 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:17:58.551062 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:17:58.551062 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:17:58.551865 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:17:58.551865 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:17:58.551865 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:17:58.551865 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:17:58.551865 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 10:17:58.551865 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 10:17:58.551865 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 10:17:58.551865 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 10:17:58.551865 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 10:17:58.551865 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 10:17:58.551865 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 15 10:17:58.551865 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application Jun 15 10:17:58.551865 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 82/167] 10.222.0.49 () {56 vars in 1099 bytes} [Wed Jun 15 10:17:58 2022] GET /identity/v3/roles/member => generated 84 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:17:58.554994 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a438d32c-62d2-499d-ac16-0af7cc5c8e1e None admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:17:58.613738 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278bfd71-4fd3-4139-ac38-9f9e54771926 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:17:58.684286 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df296521-1736-45d2-8b7a-ccde759d2425 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:17:58.686385 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df296521-1736-45d2-8b7a-ccde759d2425 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:17:58.686674 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df296521-1736-45d2-8b7a-ccde759d2425 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:17:58.688739 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df296521-1736-45d2-8b7a-ccde759d2425 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:17:58.735342 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a438d32c-62d2-499d-ac16-0af7cc5c8e1e None admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:17:58.780302 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bdb1da3-af31-4ea1-903f-4587d4cdf843 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:17:58.781029 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdb1da3-af31-4ea1-903f-4587d4cdf843 None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:17:58.781328 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdb1da3-af31-4ea1-903f-4587d4cdf843 None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:17:58.781616 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdb1da3-af31-4ea1-903f-4587d4cdf843 None admin] PATH_INFO: `/v3/roles` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:17:58.782381 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bdb1da3-af31-4ea1-903f-4587d4cdf843 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:17:58.782696 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bdb1da3-af31-4ea1-903f-4587d4cdf843 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:17:58.784301 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bdb1da3-af31-4ea1-903f-4587d4cdf843 None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:17:58.806916 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 83/168] 10.222.0.49 () {56 vars in 1099 bytes} [Wed Jun 15 10:17:58 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:17:58.807945 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bdb1da3-af31-4ea1-903f-4587d4cdf843 None admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:17:58.812186 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:17:58.839321 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df296521-1736-45d2-8b7a-ccde759d2425 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:17:58.841191 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df296521-1736-45d2-8b7a-ccde759d2425 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:17:58.906832 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=100 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:17:59.034513 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 86/169] 10.222.0.49 () {58 vars in 1282 bytes} [Wed Jun 15 10:17:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 614 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 15 10:17:59.035565 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df296521-1736-45d2-8b7a-ccde759d2425 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:17:59.057050 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bdb1da3-af31-4ea1-903f-4587d4cdf843 None admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:17:59.108499 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ac4e33-b0c4-46a3-a6bf-bc30be233e16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:17:59.109182 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ac4e33-b0c4-46a3-a6bf-bc30be233e16 None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:17:59.109461 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ac4e33-b0c4-46a3-a6bf-bc30be233e16 None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:17:59.109735 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ac4e33-b0c4-46a3-a6bf-bc30be233e16 None admin] PATH_INFO: `/v3/roles` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:17:59.110600 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ac4e33-b0c4-46a3-a6bf-bc30be233e16 None admin] RBAC: Adding query filter params (name=member) {{(pid=80119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:17:59.110910 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ac4e33-b0c4-46a3-a6bf-bc30be233e16 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:17:59.112323 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ac4e33-b0c4-46a3-a6bf-bc30be233e16 None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:17:59.121240 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 84/170] 10.222.0.49 () {56 vars in 1101 bytes} [Wed Jun 15 10:17:58 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 314 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:17:59.122013 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ac4e33-b0c4-46a3-a6bf-bc30be233e16 None admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:17:59.222338 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:17:59.235813 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ac4e33-b0c4-46a3-a6bf-bc30be233e16 None admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:17:59.238717 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=100 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:17:59.257264 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df296521-1736-45d2-8b7a-ccde759d2425 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:17:59.274796 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd92bd1-6d99-4085-b3dc-221d2d993337 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:17:59.275727 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd92bd1-6d99-4085-b3dc-221d2d993337 None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:17:59.276479 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd92bd1-6d99-4085-b3dc-221d2d993337 None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:17:59.276763 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd92bd1-6d99-4085-b3dc-221d2d993337 None admin] PATH_INFO: `/v3/projects/5f3cd857995e418386bb22cdf90c1ae9` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:17:59.277963 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd92bd1-6d99-4085-b3dc-221d2d993337 None admin] RBAC: Authorizing `identity:get_project(project_id=5f3cd857995e418386bb22cdf90c1ae9)` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:17:59.280406 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd92bd1-6d99-4085-b3dc-221d2d993337 None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:17:59.282119 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 85/171] 10.222.0.49 () {56 vars in 1186 bytes} [Wed Jun 15 10:17:59 2022] GET /identity/v3/projects/5f3cd857995e418386bb22cdf90c1ae9 => generated 298 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:17:59.283632 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5d6988-0f09-4ba1-9710-b8fd9d56c94a None None] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:17:59.283910 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5d6988-0f09-4ba1-9710-b8fd9d56c94a None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:17:59.284196 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5d6988-0f09-4ba1-9710-b8fd9d56c94a None None] PATH_INFO: `/` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:17:59.285607 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 86/172] 10.222.0.49 () {54 vars in 926 bytes} [Wed Jun 15 10:17:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:17:59.286120 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5d6988-0f09-4ba1-9710-b8fd9d56c94a None None] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:17:59.349356 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06fe4ceb-ea5b-4aa9-831f-ea26f06b877d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:17:59.350268 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fe4ceb-ea5b-4aa9-831f-ea26f06b877d None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:17:59.350550 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fe4ceb-ea5b-4aa9-831f-ea26f06b877d None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:17:59.350840 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fe4ceb-ea5b-4aa9-831f-ea26f06b877d None admin] PATH_INFO: `/v3/users/fcdb2d8e71bd4fe6b9b1431183facba7` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:17:59.352172 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06fe4ceb-ea5b-4aa9-831f-ea26f06b877d None admin] RBAC: Authorizing `identity:get_user(user_id=fcdb2d8e71bd4fe6b9b1431183facba7)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:17:59.353673 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06fe4ceb-ea5b-4aa9-831f-ea26f06b877d None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:17:59.355669 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 87/173] 10.222.0.49 () {56 vars in 1177 bytes} [Wed Jun 15 10:17:59 2022] GET /identity/v3/users/fcdb2d8e71bd4fe6b9b1431183facba7 => generated 247 bytes in 320 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:17:59.356303 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06fe4ceb-ea5b-4aa9-831f-ea26f06b877d None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:17:59.433560 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5d6988-0f09-4ba1-9710-b8fd9d56c94a None None] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:17:59.472705 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4fa870-fcef-44ad-b8ac-48f2a1d29f0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:17:59.473592 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4fa870-fcef-44ad-b8ac-48f2a1d29f0b None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:17:59.473878 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4fa870-fcef-44ad-b8ac-48f2a1d29f0b None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:17:59.474154 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4fa870-fcef-44ad-b8ac-48f2a1d29f0b None admin] PATH_INFO: `/v3/users/2f093b1bbc6148f39242d0692e1e8a95` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:17:59.477202 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4fa870-fcef-44ad-b8ac-48f2a1d29f0b None admin] RBAC: Authorizing `identity:get_user(user_id=2f093b1bbc6148f39242d0692e1e8a95)` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:17:59.479137 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4fa870-fcef-44ad-b8ac-48f2a1d29f0b None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:17:59.480873 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 87/174] 10.222.0.49 () {56 vars in 1177 bytes} [Wed Jun 15 10:17:59 2022] GET /identity/v3/users/2f093b1bbc6148f39242d0692e1e8a95 => generated 275 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:17:59.482154 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0deb64-b613-4b08-b901-1c3fcc370552 None None] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:17:59.482486 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0deb64-b613-4b08-b901-1c3fcc370552 None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:17:59.482774 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0deb64-b613-4b08-b901-1c3fcc370552 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:17:59.485820 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06fe4ceb-ea5b-4aa9-831f-ea26f06b877d None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:17:59.500789 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.auth.core [None req-8d0deb64-b613-4b08-b901-1c3fcc370552 None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:17:59.502047 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-8d0deb64-b613-4b08-b901-1c3fcc370552 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:17:59.531822 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9af350f-209f-47fc-ab9a-adfb4fdaef9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:17:59.532972 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9af350f-209f-47fc-ab9a-adfb4fdaef9b None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:17:59.533776 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9af350f-209f-47fc-ab9a-adfb4fdaef9b None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:17:59.534569 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9af350f-209f-47fc-ab9a-adfb4fdaef9b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:17:59.535915 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9af350f-209f-47fc-ab9a-adfb4fdaef9b None admin] RBAC: Adding query filter params (role.id=4d2118569d074a288b37d120110c300d, scope.project.id=5f3cd857995e418386bb22cdf90c1ae9, user.id=2f093b1bbc6148f39242d0692e1e8a95) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:17:59.536559 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9af350f-209f-47fc-ab9a-adfb4fdaef9b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:17:59.538196 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9af350f-209f-47fc-ab9a-adfb4fdaef9b None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:17:59.546133 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 88/175] 10.222.0.49 () {56 vars in 1374 bytes} [Wed Jun 15 10:17:59 2022] GET /identity/v3/role_assignments?user.id=2f093b1bbc6148f39242d0692e1e8a95&scope.project.id=5f3cd857995e418386bb22cdf90c1ae9&role.id=4d2118569d074a288b37d120110c300d => generated 608 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:17:59.547601 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9af350f-209f-47fc-ab9a-adfb4fdaef9b None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:17:59.548209 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=100 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:17:59.576789 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:17:59.610145 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 88/176] 10.222.0.49 () {58 vars in 1023 bytes} [Wed Jun 15 10:17:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 10:17:59.611432 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bd9949-bf38-4e15-bde0-bb0170bac7ee None None] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:17:59.611714 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bd9949-bf38-4e15-bde0-bb0170bac7ee None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:17:59.612019 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bd9949-bf38-4e15-bde0-bb0170bac7ee None None] PATH_INFO: `/` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:17:59.613461 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 89/177] 10.222.0.49 () {54 vars in 926 bytes} [Wed Jun 15 10:17:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:17:59.614029 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5bd9949-bf38-4e15-bde0-bb0170bac7ee None None] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:17:59.686661 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9af350f-209f-47fc-ab9a-adfb4fdaef9b None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:17:59.733409 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2156d3-8a08-4cc9-bbfd-15a90ee96772 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:17:59.734475 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2156d3-8a08-4cc9-bbfd-15a90ee96772 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:17:59.734953 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2156d3-8a08-4cc9-bbfd-15a90ee96772 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:17:59.735439 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2156d3-8a08-4cc9-bbfd-15a90ee96772 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:17:59.815324 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5bd9949-bf38-4e15-bde0-bb0170bac7ee None None] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:17:59.841987 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b2156d3-8a08-4cc9-bbfd-15a90ee96772 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:17:59.844096 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b2156d3-8a08-4cc9-bbfd-15a90ee96772 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:17:59.866501 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:17:59.869116 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee57daca-b902-4971-b6f8-de82a28b4a1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:17:59.869116 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee57daca-b902-4971-b6f8-de82a28b4a1e None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:17:59.869116 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee57daca-b902-4971-b6f8-de82a28b4a1e None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:17:59.869116 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee57daca-b902-4971-b6f8-de82a28b4a1e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:17:59.913986 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=100 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:17:59.943001 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee57daca-b902-4971-b6f8-de82a28b4a1e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:17:59.944529 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee57daca-b902-4971-b6f8-de82a28b4a1e None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:17:59.950902 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 89/178] 10.222.0.49 () {58 vars in 1282 bytes} [Wed Jun 15 10:17:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 404 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 15 10:17:59.952926 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4948592e-21c0-47f0-936e-9655dc02f9db None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:17:59.953226 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4948592e-21c0-47f0-936e-9655dc02f9db None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:17:59.953507 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4948592e-21c0-47f0-936e-9655dc02f9db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:17:59.969410 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-4948592e-21c0-47f0-936e-9655dc02f9db None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:17:59.970601 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-4948592e-21c0-47f0-936e-9655dc02f9db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:18:00.036087 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 90/179] 10.222.0.49 () {58 vars in 1023 bytes} [Wed Jun 15 10:17:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 10:18:00.036925 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4948592e-21c0-47f0-936e-9655dc02f9db None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:18:00.054648 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 90/180] 10.222.0.49 () {58 vars in 1282 bytes} [Wed Jun 15 10:17:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 441 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 15 10:18:00.060295 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3211113e-c5fd-4a10-bb14-e82a915d4e7c None None] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:00.060588 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3211113e-c5fd-4a10-bb14-e82a915d4e7c None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:00.060861 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3211113e-c5fd-4a10-bb14-e82a915d4e7c None None] PATH_INFO: `/` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:00.062165 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 91/181] 10.222.0.49 () {54 vars in 926 bytes} [Wed Jun 15 10:18:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:18:00.063255 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f56d6aa-54f0-460e-88a6-5583cbfd7ec3 None None] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:00.063635 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f56d6aa-54f0-460e-88a6-5583cbfd7ec3 None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:00.063804 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f56d6aa-54f0-460e-88a6-5583cbfd7ec3 None None] PATH_INFO: `/` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:00.065118 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 92/182] 10.222.0.49 () {54 vars in 926 bytes} [Wed Jun 15 10:18:00 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:18:00.067868 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f56d6aa-54f0-460e-88a6-5583cbfd7ec3 None None] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:18:00.086030 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:18:00.134978 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4948592e-21c0-47f0-936e-9655dc02f9db None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:18:00.136840 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=100 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:18:00.168857 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b23350-89a6-43b4-b516-140346974367 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:18:00.169378 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f56d6aa-54f0-460e-88a6-5583cbfd7ec3 None None] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:18:00.169658 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b23350-89a6-43b4-b516-140346974367 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:00.169968 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b23350-89a6-43b4-b516-140346974367 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:00.170240 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b23350-89a6-43b4-b516-140346974367 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:00.180379 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b23350-89a6-43b4-b516-140346974367 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 10:18:00.188151 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application [None req-24b23350-89a6-43b4-b516-140346974367 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 10:18:00.188151 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:18:00.188151 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:18:00.188151 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:18:00.188151 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:18:00.188151 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:18:00.188151 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:18:00.188151 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:18:00.188151 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:18:00.188151 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:18:00.188151 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:18:00.188151 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:18:00.188151 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 10:18:00.188151 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 10:18:00.188151 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 10:18:00.188151 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 10:18:00.189148 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 10:18:00.189148 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 10:18:00.189148 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 10:18:00.189148 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 10:18:00.189148 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:18:00.189148 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:18:00.189148 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:18:00.189148 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:18:00.189148 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:18:00.189148 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:18:00.189148 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:18:00.189148 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application with Lock( Jun 15 10:18:00.189148 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:18:00.189148 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:18:00.189148 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:18:00.189148 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:18:00.189950 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:18:00.189950 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:18:00.189950 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:18:00.189950 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:18:00.189950 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 10:18:00.189950 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 10:18:00.189950 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 10:18:00.189950 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 10:18:00.189950 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 10:18:00.189950 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 10:18:00.189950 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 10:18:00.189950 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Jun 15 10:18:00.189950 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 91/183] 10.222.0.49 () {56 vars in 1096 bytes} [Wed Jun 15 10:18:00 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:18:00.190605 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b23350-89a6-43b4-b516-140346974367 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:18:00.206024 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a60d02-b107-4d19-bd7c-c93d93421b73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:18:00.206919 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a60d02-b107-4d19-bd7c-c93d93421b73 None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:00.207240 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a60d02-b107-4d19-bd7c-c93d93421b73 None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:00.207520 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a60d02-b107-4d19-bd7c-c93d93421b73 None admin] PATH_INFO: `/v3/projects/5f3cd857995e418386bb22cdf90c1ae9` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:00.208705 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a60d02-b107-4d19-bd7c-c93d93421b73 None admin] RBAC: Authorizing `identity:get_project(project_id=5f3cd857995e418386bb22cdf90c1ae9)` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:18:00.210168 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a60d02-b107-4d19-bd7c-c93d93421b73 None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:18:00.211907 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 93/184] 10.222.0.49 () {56 vars in 1186 bytes} [Wed Jun 15 10:18:00 2022] GET /identity/v3/projects/5f3cd857995e418386bb22cdf90c1ae9 => generated 298 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:18:00.212668 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a60d02-b107-4d19-bd7c-c93d93421b73 None admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:18:00.296801 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b23350-89a6-43b4-b516-140346974367 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:18:00.306963 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:18:00.310881 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a60d02-b107-4d19-bd7c-c93d93421b73 None admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:18:00.331198 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9220cb-2379-4ea0-a4a6-9a63f00dae68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:18:00.332015 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9220cb-2379-4ea0-a4a6-9a63f00dae68 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:00.332301 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9220cb-2379-4ea0-a4a6-9a63f00dae68 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:00.332577 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9220cb-2379-4ea0-a4a6-9a63f00dae68 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:00.343034 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9220cb-2379-4ea0-a4a6-9a63f00dae68 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 10:18:00.345091 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533c5a71-8ad2-49ee-b660-d91f3533282e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:18:00.345954 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533c5a71-8ad2-49ee-b660-d91f3533282e None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:00.346238 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533c5a71-8ad2-49ee-b660-d91f3533282e None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:00.346512 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533c5a71-8ad2-49ee-b660-d91f3533282e None admin] PATH_INFO: `/v3/projects/24ea2a1e0d1f49bebb8a85e96ebbe085` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:00.347721 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533c5a71-8ad2-49ee-b660-d91f3533282e None admin] RBAC: Authorizing `identity:get_project(project_id=24ea2a1e0d1f49bebb8a85e96ebbe085)` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:18:00.349183 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533c5a71-8ad2-49ee-b660-d91f3533282e None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:18:00.351122 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 94/185] 10.222.0.49 () {56 vars in 1186 bytes} [Wed Jun 15 10:18:00 2022] GET /identity/v3/projects/24ea2a1e0d1f49bebb8a85e96ebbe085 => generated 312 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:18:00.351224 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application [None req-9f9220cb-2379-4ea0-a4a6-9a63f00dae68 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 15 10:18:00.351224 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:18:00.351224 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:18:00.351224 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:18:00.351224 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:18:00.351224 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:18:00.351224 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:18:00.351224 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:18:00.351224 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:18:00.351224 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:18:00.351224 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:18:00.351224 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:18:00.351224 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 10:18:00.351224 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 10:18:00.351224 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 10:18:00.351224 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 10:18:00.352344 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 10:18:00.352344 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 10:18:00.352344 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 10:18:00.352344 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 10:18:00.352344 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:18:00.352344 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:18:00.352344 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:18:00.352344 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:18:00.352344 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:18:00.352344 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:18:00.352344 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:18:00.352344 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application with Lock( Jun 15 10:18:00.352344 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:18:00.352344 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:18:00.352344 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:18:00.352344 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:18:00.353122 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:18:00.353122 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:18:00.353122 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:18:00.353122 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:18:00.353122 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 10:18:00.353122 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 10:18:00.353122 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 10:18:00.353122 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 10:18:00.353122 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 10:18:00.353122 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 10:18:00.353122 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 15 10:18:00.353122 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Jun 15 10:18:00.353716 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee5139d-f2eb-414e-a463-63d267bb7d93 None None] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:00.353783 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 92/186] 10.222.0.49 () {56 vars in 1114 bytes} [Wed Jun 15 10:18:00 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:18:00.353856 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee5139d-f2eb-414e-a463-63d267bb7d93 None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:00.353856 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee5139d-f2eb-414e-a463-63d267bb7d93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:00.353976 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbc3691-18d1-4140-854d-1ac2d2065171 None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:00.353976 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbc3691-18d1-4140-854d-1ac2d2065171 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:00.353976 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbc3691-18d1-4140-854d-1ac2d2065171 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:00.365233 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=100 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:18:00.370776 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-dfbc3691-18d1-4140-854d-1ac2d2065171 None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:18:00.371298 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.auth.core [None req-bee5139d-f2eb-414e-a463-63d267bb7d93 None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:18:00.372012 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-dfbc3691-18d1-4140-854d-1ac2d2065171 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:18:00.372456 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-bee5139d-f2eb-414e-a463-63d267bb7d93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:18:00.438147 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 95/187] 10.222.0.49 () {58 vars in 1023 bytes} [Wed Jun 15 10:18:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 10:18:00.439148 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee5139d-f2eb-414e-a463-63d267bb7d93 None None] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:18:00.443226 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 93/188] 10.222.0.49 () {58 vars in 1023 bytes} [Wed Jun 15 10:18:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 10:18:00.444312 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfbc3691-18d1-4140-854d-1ac2d2065171 None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:18:00.536380 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee5139d-f2eb-414e-a463-63d267bb7d93 None None] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:18:00.551927 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:18:00.555065 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfbc3691-18d1-4140-854d-1ac2d2065171 None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:18:00.570398 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b510ed7-196d-4aca-ad9d-228a6d14be05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:18:00.571016 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b510ed7-196d-4aca-ad9d-228a6d14be05 None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:00.571325 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b510ed7-196d-4aca-ad9d-228a6d14be05 None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:00.571608 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b510ed7-196d-4aca-ad9d-228a6d14be05 None admin] PATH_INFO: `/v3/roles` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:00.572367 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b510ed7-196d-4aca-ad9d-228a6d14be05 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:18:00.572678 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b510ed7-196d-4aca-ad9d-228a6d14be05 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:18:00.574110 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b510ed7-196d-4aca-ad9d-228a6d14be05 None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:18:00.577234 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=100 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:18:00.584570 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 96/189] 10.222.0.49 () {56 vars in 1099 bytes} [Wed Jun 15 10:18:00 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:18:00.589126 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b510ed7-196d-4aca-ad9d-228a6d14be05 None admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:18:00.594183 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f98498-bc23-47b1-b253-ec25c08f8b6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:18:00.594839 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f98498-bc23-47b1-b253-ec25c08f8b6d None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:00.595155 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f98498-bc23-47b1-b253-ec25c08f8b6d None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:00.595434 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f98498-bc23-47b1-b253-ec25c08f8b6d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:00.596265 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f98498-bc23-47b1-b253-ec25c08f8b6d None admin] RBAC: Adding query filter params (role.id=8051174e7a7947d5b991ebd793c2e543, scope.project.id=5f3cd857995e418386bb22cdf90c1ae9, user.id=fcdb2d8e71bd4fe6b9b1431183facba7) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:18:00.596571 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f98498-bc23-47b1-b253-ec25c08f8b6d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:18:00.598023 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f98498-bc23-47b1-b253-ec25c08f8b6d None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:18:00.603846 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 94/190] 10.222.0.49 () {56 vars in 1374 bytes} [Wed Jun 15 10:18:00 2022] GET /identity/v3/role_assignments?user.id=fcdb2d8e71bd4fe6b9b1431183facba7&scope.project.id=5f3cd857995e418386bb22cdf90c1ae9&role.id=8051174e7a7947d5b991ebd793c2e543 => generated 608 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:18:00.604670 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f98498-bc23-47b1-b253-ec25c08f8b6d None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:18:00.706462 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b510ed7-196d-4aca-ad9d-228a6d14be05 None admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:18:00.720669 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f98498-bc23-47b1-b253-ec25c08f8b6d None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:18:00.743079 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b633aa-31c5-4be0-99cb-abc95fb4cd85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:18:00.743782 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b633aa-31c5-4be0-99cb-abc95fb4cd85 None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:00.744068 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b633aa-31c5-4be0-99cb-abc95fb4cd85 None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:00.744341 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b633aa-31c5-4be0-99cb-abc95fb4cd85 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:00.745191 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b633aa-31c5-4be0-99cb-abc95fb4cd85 None admin] RBAC: Adding query filter params (role.id=6876e57cc51c448097ab109d35b2a3da, scope.project.id=24ea2a1e0d1f49bebb8a85e96ebbe085, user.id=2f093b1bbc6148f39242d0692e1e8a95) {{(pid=80119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:18:00.745507 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b633aa-31c5-4be0-99cb-abc95fb4cd85 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:18:00.747283 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b633aa-31c5-4be0-99cb-abc95fb4cd85 None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:18:00.759271 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 97/191] 10.222.0.49 () {56 vars in 1374 bytes} [Wed Jun 15 10:18:00 2022] GET /identity/v3/role_assignments?user.id=2f093b1bbc6148f39242d0692e1e8a95&scope.project.id=24ea2a1e0d1f49bebb8a85e96ebbe085&role.id=6876e57cc51c448097ab109d35b2a3da => generated 608 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:18:00.759271 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b633aa-31c5-4be0-99cb-abc95fb4cd85 None admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:18:00.768631 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d1b40e-bfde-4f36-971a-2fc27433070f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:18:00.771632 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d1b40e-bfde-4f36-971a-2fc27433070f None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:00.771632 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d1b40e-bfde-4f36-971a-2fc27433070f None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:00.771632 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d1b40e-bfde-4f36-971a-2fc27433070f None admin] PATH_INFO: `/v3/roles` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:00.771632 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d1b40e-bfde-4f36-971a-2fc27433070f None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:18:00.771632 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d1b40e-bfde-4f36-971a-2fc27433070f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:18:00.772895 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d1b40e-bfde-4f36-971a-2fc27433070f None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:18:00.783714 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 95/192] 10.222.0.49 () {56 vars in 1111 bytes} [Wed Jun 15 10:18:00 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:18:00.784665 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d1b40e-bfde-4f36-971a-2fc27433070f None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:18:00.829811 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:18:00.853202 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=100 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:18:00.905992 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b633aa-31c5-4be0-99cb-abc95fb4cd85 None admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:18:00.940992 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d1b40e-bfde-4f36-971a-2fc27433070f None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:18:00.945462 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e60a8a4-6864-46ba-ba72-56c537bcb6d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:18:00.946366 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e60a8a4-6864-46ba-ba72-56c537bcb6d3 None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:00.946661 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e60a8a4-6864-46ba-ba72-56c537bcb6d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:00.946943 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e60a8a4-6864-46ba-ba72-56c537bcb6d3 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:00.960031 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e60a8a4-6864-46ba-ba72-56c537bcb6d3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 10:18:00.968853 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application [None req-1e60a8a4-6864-46ba-ba72-56c537bcb6d3 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 15 10:18:00.968853 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:18:00.968853 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:18:00.968853 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:18:00.968853 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:18:00.968853 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:18:00.968853 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:18:00.968853 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:18:00.968853 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:18:00.968853 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:18:00.968853 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:18:00.968853 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:18:00.968853 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 10:18:00.968853 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 10:18:00.968853 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 10:18:00.968853 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 10:18:00.969934 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 10:18:00.969934 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 10:18:00.969934 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 10:18:00.969934 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 10:18:00.969934 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:18:00.969934 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:18:00.969934 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:18:00.969934 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:18:00.969934 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:18:00.969934 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:18:00.969934 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:18:00.969934 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application with Lock( Jun 15 10:18:00.969934 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:18:00.969934 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:18:00.969934 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:18:00.969934 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:18:00.970729 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:18:00.970729 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:18:00.970729 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:18:00.970729 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:18:00.970729 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 10:18:00.970729 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 10:18:00.970729 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 10:18:00.970729 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 10:18:00.970729 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 10:18:00.970729 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 10:18:00.970729 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 15 10:18:00.970729 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application Jun 15 10:18:00.970729 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 98/193] 10.222.0.49 () {56 vars in 1099 bytes} [Wed Jun 15 10:18:00 2022] GET /identity/v3/roles/reader => generated 84 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:18:00.971442 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e60a8a4-6864-46ba-ba72-56c537bcb6d3 None admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:18:00.982292 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdaf8b8f-e7b6-4819-8870-faf0eb79aa8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:18:00.983858 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdaf8b8f-e7b6-4819-8870-faf0eb79aa8d None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:00.984147 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdaf8b8f-e7b6-4819-8870-faf0eb79aa8d None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:00.984427 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdaf8b8f-e7b6-4819-8870-faf0eb79aa8d None admin] PATH_INFO: `/v3/domains/default` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:00.985690 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdaf8b8f-e7b6-4819-8870-faf0eb79aa8d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:18:00.987169 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdaf8b8f-e7b6-4819-8870-faf0eb79aa8d None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:18:00.989292 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 96/194] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Jun 15 10:18:00 2022] GET /identity/v3/domains/default => generated 199 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:18:00.990110 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdaf8b8f-e7b6-4819-8870-faf0eb79aa8d None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:18:01.075494 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e60a8a4-6864-46ba-ba72-56c537bcb6d3 None admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:18:01.082226 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:18:01.088923 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdaf8b8f-e7b6-4819-8870-faf0eb79aa8d None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:18:01.097715 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=100 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:18:01.110435 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87cca4a8-41fd-4cef-9a7f-8353e2d0e7d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:18:01.111344 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cca4a8-41fd-4cef-9a7f-8353e2d0e7d7 None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:01.111623 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cca4a8-41fd-4cef-9a7f-8353e2d0e7d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:01.111899 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cca4a8-41fd-4cef-9a7f-8353e2d0e7d7 None admin] PATH_INFO: `/v3/users/c3f14bfbae9a41858def110ed6fe1d8e` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:01.121874 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f733b6f-b662-4a8e-a74c-b597d2ae32c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:18:01.122748 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f733b6f-b662-4a8e-a74c-b597d2ae32c7 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:01.123096 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f733b6f-b662-4a8e-a74c-b597d2ae32c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:01.123885 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f733b6f-b662-4a8e-a74c-b597d2ae32c7 None admin] PATH_INFO: `/v3/users/c3f14bfbae9a41858def110ed6fe1d8e` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:01.125837 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87cca4a8-41fd-4cef-9a7f-8353e2d0e7d7 None admin] RBAC: Authorizing `identity:get_user(user_id=c3f14bfbae9a41858def110ed6fe1d8e)` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:18:01.127326 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87cca4a8-41fd-4cef-9a7f-8353e2d0e7d7 None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:18:01.128976 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 99/195] 10.222.0.49 () {56 vars in 1177 bytes} [Wed Jun 15 10:18:00 2022] GET /identity/v3/users/c3f14bfbae9a41858def110ed6fe1d8e => generated 283 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:18:01.129741 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87cca4a8-41fd-4cef-9a7f-8353e2d0e7d7 None admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:18:01.139487 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f733b6f-b662-4a8e-a74c-b597d2ae32c7 None admin] RBAC: Authorizing `identity:get_user(user_id=c3f14bfbae9a41858def110ed6fe1d8e)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:18:01.140999 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f733b6f-b662-4a8e-a74c-b597d2ae32c7 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:18:01.142701 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 97/196] 10.222.0.49 () {56 vars in 1177 bytes} [Wed Jun 15 10:18:00 2022] GET /identity/v3/users/c3f14bfbae9a41858def110ed6fe1d8e => generated 283 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:18:01.143449 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f733b6f-b662-4a8e-a74c-b597d2ae32c7 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:18:01.228409 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87cca4a8-41fd-4cef-9a7f-8353e2d0e7d7 None admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:18:01.243473 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f733b6f-b662-4a8e-a74c-b597d2ae32c7 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:18:01.262221 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701f661b-c018-4671-bb1d-16def5d88226 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:18:01.262890 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701f661b-c018-4671-bb1d-16def5d88226 None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:01.263201 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701f661b-c018-4671-bb1d-16def5d88226 None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:01.263482 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701f661b-c018-4671-bb1d-16def5d88226 None admin] PATH_INFO: `/v3/roles` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:01.264224 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701f661b-c018-4671-bb1d-16def5d88226 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:18:01.264532 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701f661b-c018-4671-bb1d-16def5d88226 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:18:01.265923 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701f661b-c018-4671-bb1d-16def5d88226 None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:18:01.274281 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 100/197] 10.222.0.49 () {56 vars in 1101 bytes} [Wed Jun 15 10:18:01 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:18:01.274908 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701f661b-c018-4671-bb1d-16def5d88226 None admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:18:01.277552 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2413f6f9-699e-45e1-8172-f99f84d9e1ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:18:01.278234 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2413f6f9-699e-45e1-8172-f99f84d9e1ae None admin] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:01.278534 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2413f6f9-699e-45e1-8172-f99f84d9e1ae None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:01.278832 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2413f6f9-699e-45e1-8172-f99f84d9e1ae None admin] PATH_INFO: `/v3/users` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:01.280125 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2413f6f9-699e-45e1-8172-f99f84d9e1ae None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:18:01.281566 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2413f6f9-699e-45e1-8172-f99f84d9e1ae None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:18:01.301588 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 98/198] 10.222.0.49 () {60 vars in 1132 bytes} [Wed Jun 15 10:18:01 2022] POST /identity/v3/users => generated 297 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 10:18:01.302406 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2413f6f9-699e-45e1-8172-f99f84d9e1ae None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:18:01.315081 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:18:01.336908 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=100 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:18:01.406348 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701f661b-c018-4671-bb1d-16def5d88226 None admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:18:01.418953 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2413f6f9-699e-45e1-8172-f99f84d9e1ae None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:18:01.444834 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28225347-f341-49d2-ac66-8c877a8c3f37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:18:01.445487 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28225347-f341-49d2-ac66-8c877a8c3f37 None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:01.445778 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28225347-f341-49d2-ac66-8c877a8c3f37 None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:01.446058 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28225347-f341-49d2-ac66-8c877a8c3f37 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:01.456081 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0756cbbd-de0b-4b52-92cb-0c038efae046 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:18:01.457038 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0756cbbd-de0b-4b52-92cb-0c038efae046 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:01.457142 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0756cbbd-de0b-4b52-92cb-0c038efae046 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:01.457386 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0756cbbd-de0b-4b52-92cb-0c038efae046 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:01.526282 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28225347-f341-49d2-ac66-8c877a8c3f37 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:18:01.527809 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28225347-f341-49d2-ac66-8c877a8c3f37 None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:18:01.550398 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0756cbbd-de0b-4b52-92cb-0c038efae046 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:18:01.552246 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0756cbbd-de0b-4b52-92cb-0c038efae046 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:18:01.566131 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:18:01.578829 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=100 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:18:01.649168 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 101/199] 10.222.0.49 () {58 vars in 1282 bytes} [Wed Jun 15 10:18:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 15 10:18:01.650445 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28225347-f341-49d2-ac66-8c877a8c3f37 None admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:18:01.674445 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 99/200] 10.222.0.49 () {58 vars in 1282 bytes} [Wed Jun 15 10:18:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 15 10:18:01.675806 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0756cbbd-de0b-4b52-92cb-0c038efae046 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:18:01.767482 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28225347-f341-49d2-ac66-8c877a8c3f37 None admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:18:01.789265 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0756cbbd-de0b-4b52-92cb-0c038efae046 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:18:01.799695 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:18:01.803577 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee79e3dc-aa05-4854-838b-b969f098af6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:18:01.804464 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee79e3dc-aa05-4854-838b-b969f098af6d None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:01.804748 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee79e3dc-aa05-4854-838b-b969f098af6d None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:01.805024 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee79e3dc-aa05-4854-838b-b969f098af6d None admin] PATH_INFO: `/v3/users/dfe1d4745a4d4642a21f8860b4a02fb2` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:01.806244 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee79e3dc-aa05-4854-838b-b969f098af6d None admin] RBAC: Authorizing `identity:get_user(user_id=dfe1d4745a4d4642a21f8860b4a02fb2)` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:18:01.807814 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee79e3dc-aa05-4854-838b-b969f098af6d None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:18:01.814150 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 102/201] 10.222.0.49 () {56 vars in 1177 bytes} [Wed Jun 15 10:18:01 2022] GET /identity/v3/users/dfe1d4745a4d4642a21f8860b4a02fb2 => generated 289 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:18:01.815025 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee79e3dc-aa05-4854-838b-b969f098af6d None admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:18:01.828284 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b58a7e-4d14-44e4-aae9-8633553d442e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:18:01.828629 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=100 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:18:01.829778 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b58a7e-4d14-44e4-aae9-8633553d442e None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:01.830064 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b58a7e-4d14-44e4-aae9-8633553d442e None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:01.830329 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b58a7e-4d14-44e4-aae9-8633553d442e None admin] PATH_INFO: `/v3/projects/df1f045b58c24b4dba0964eff7d2e540` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:01.831773 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b58a7e-4d14-44e4-aae9-8633553d442e None admin] RBAC: Authorizing `identity:get_project(project_id=df1f045b58c24b4dba0964eff7d2e540)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:18:01.833315 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b58a7e-4d14-44e4-aae9-8633553d442e None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:18:01.835058 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 100/202] 10.222.0.49 () {56 vars in 1186 bytes} [Wed Jun 15 10:18:01 2022] GET /identity/v3/projects/df1f045b58c24b4dba0964eff7d2e540 => generated 302 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:18:01.836038 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b58a7e-4d14-44e4-aae9-8633553d442e None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:18:01.937512 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee79e3dc-aa05-4854-838b-b969f098af6d None admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:18:01.959898 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b58a7e-4d14-44e4-aae9-8633553d442e None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:18:01.972687 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a87db69-2d48-44c3-b72f-2bb74e7003dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:18:01.973563 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a87db69-2d48-44c3-b72f-2bb74e7003dd None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:01.973842 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a87db69-2d48-44c3-b72f-2bb74e7003dd None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:01.974114 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a87db69-2d48-44c3-b72f-2bb74e7003dd None admin] PATH_INFO: `/v3/projects/df1f045b58c24b4dba0964eff7d2e540` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:01.975343 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a87db69-2d48-44c3-b72f-2bb74e7003dd None admin] RBAC: Authorizing `identity:get_project(project_id=df1f045b58c24b4dba0964eff7d2e540)` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:18:01.976872 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a87db69-2d48-44c3-b72f-2bb74e7003dd None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:18:01.978877 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 103/203] 10.222.0.49 () {56 vars in 1186 bytes} [Wed Jun 15 10:18:01 2022] GET /identity/v3/projects/df1f045b58c24b4dba0964eff7d2e540 => generated 302 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:18:01.979784 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a87db69-2d48-44c3-b72f-2bb74e7003dd None admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:18:01.999483 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cafc67b-a7a2-4374-9927-7d6744ca5921 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:18:02.000148 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cafc67b-a7a2-4374-9927-7d6744ca5921 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:02.000431 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cafc67b-a7a2-4374-9927-7d6744ca5921 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:02.000705 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cafc67b-a7a2-4374-9927-7d6744ca5921 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:02.050982 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:18:02.062265 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=100 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:18:02.081099 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cafc67b-a7a2-4374-9927-7d6744ca5921 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:18:02.082932 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cafc67b-a7a2-4374-9927-7d6744ca5921 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:18:02.084584 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a87db69-2d48-44c3-b72f-2bb74e7003dd None admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:18:02.118857 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb94ecb-df35-40f3-89bd-9d6c791693d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:18:02.119549 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb94ecb-df35-40f3-89bd-9d6c791693d0 None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:02.119830 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb94ecb-df35-40f3-89bd-9d6c791693d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:02.120107 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb94ecb-df35-40f3-89bd-9d6c791693d0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:02.121373 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb94ecb-df35-40f3-89bd-9d6c791693d0 None admin] RBAC: Adding query filter params (role.id=8051174e7a7947d5b991ebd793c2e543, scope.project.id=df1f045b58c24b4dba0964eff7d2e540, user.id=c3f14bfbae9a41858def110ed6fe1d8e) {{(pid=80119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:18:02.121373 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb94ecb-df35-40f3-89bd-9d6c791693d0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:18:02.122708 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb94ecb-df35-40f3-89bd-9d6c791693d0 None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:18:02.129314 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 104/204] 10.222.0.49 () {56 vars in 1374 bytes} [Wed Jun 15 10:18:01 2022] GET /identity/v3/role_assignments?user.id=c3f14bfbae9a41858def110ed6fe1d8e&scope.project.id=df1f045b58c24b4dba0964eff7d2e540&role.id=8051174e7a7947d5b991ebd793c2e543 => generated 259 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:18:02.136126 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb94ecb-df35-40f3-89bd-9d6c791693d0 None admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:18:02.203759 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 101/205] 10.222.0.49 () {58 vars in 1282 bytes} [Wed Jun 15 10:18:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 15 10:18:02.210289 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cafc67b-a7a2-4374-9927-7d6744ca5921 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:18:02.332516 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:18:02.359996 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cafc67b-a7a2-4374-9927-7d6744ca5921 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:18:02.363406 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb94ecb-df35-40f3-89bd-9d6c791693d0 None admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:18:02.400147 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5c266b-5eec-4668-a6ae-b5a1b027e721 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:18:02.401349 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5c266b-5eec-4668-a6ae-b5a1b027e721 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:02.401851 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5c266b-5eec-4668-a6ae-b5a1b027e721 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:02.402317 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5c266b-5eec-4668-a6ae-b5a1b027e721 None admin] PATH_INFO: `/v3/projects/5f3cd857995e418386bb22cdf90c1ae9` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:02.403726 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b5c266b-5eec-4668-a6ae-b5a1b027e721 None admin] RBAC: Authorizing `identity:get_project(project_id=5f3cd857995e418386bb22cdf90c1ae9)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:18:02.404494 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44297ef0-8652-448b-886a-7353bc78e643 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:18:02.405156 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44297ef0-8652-448b-886a-7353bc78e643 None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:02.405442 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44297ef0-8652-448b-886a-7353bc78e643 None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:02.405717 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44297ef0-8652-448b-886a-7353bc78e643 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:02.406298 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b5c266b-5eec-4668-a6ae-b5a1b027e721 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:18:02.406540 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44297ef0-8652-448b-886a-7353bc78e643 None admin] RBAC: Adding query filter params (role.id=4d2118569d074a288b37d120110c300d, scope.project.id=df1f045b58c24b4dba0964eff7d2e540, user.id=c3f14bfbae9a41858def110ed6fe1d8e) {{(pid=80119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:18:02.406851 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44297ef0-8652-448b-886a-7353bc78e643 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:18:02.408318 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44297ef0-8652-448b-886a-7353bc78e643 None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:18:02.408959 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 102/206] 10.222.0.49 () {56 vars in 1186 bytes} [Wed Jun 15 10:18:02 2022] GET /identity/v3/projects/5f3cd857995e418386bb22cdf90c1ae9 => generated 298 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 10:18:02.414344 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 105/207] 10.222.0.49 () {56 vars in 1374 bytes} [Wed Jun 15 10:18:02 2022] GET /identity/v3/role_assignments?user.id=c3f14bfbae9a41858def110ed6fe1d8e&scope.project.id=df1f045b58c24b4dba0964eff7d2e540&role.id=4d2118569d074a288b37d120110c300d => generated 259 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:18:02.417004 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5c266b-5eec-4668-a6ae-b5a1b027e721 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:18:02.548328 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5c266b-5eec-4668-a6ae-b5a1b027e721 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:18:02.589744 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f95986-3222-4441-9b9e-1bdcd83c15f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:18:02.591275 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f95986-3222-4441-9b9e-1bdcd83c15f7 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:02.591859 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f95986-3222-4441-9b9e-1bdcd83c15f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:02.592408 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f95986-3222-4441-9b9e-1bdcd83c15f7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:02.593512 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f95986-3222-4441-9b9e-1bdcd83c15f7 None admin] RBAC: Adding query filter params (role.id=301a3c8a23ba40edbc35df1da8af0de9, scope.project.id=5f3cd857995e418386bb22cdf90c1ae9, user.id=dfe1d4745a4d4642a21f8860b4a02fb2) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:18:02.594049 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f95986-3222-4441-9b9e-1bdcd83c15f7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:18:02.595724 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f95986-3222-4441-9b9e-1bdcd83c15f7 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:18:02.602766 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 103/208] 10.222.0.49 () {56 vars in 1374 bytes} [Wed Jun 15 10:18:02 2022] GET /identity/v3/role_assignments?user.id=dfe1d4745a4d4642a21f8860b4a02fb2&scope.project.id=5f3cd857995e418386bb22cdf90c1ae9&role.id=301a3c8a23ba40edbc35df1da8af0de9 => generated 608 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:18:03.665562 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aad58ab-cf8b-4267-a578-5362eda87eea None None] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:03.666110 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aad58ab-cf8b-4267-a578-5362eda87eea None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:03.666110 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aad58ab-cf8b-4267-a578-5362eda87eea None None] PATH_INFO: `/` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:03.667824 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 106/209] 10.222.0.49 () {54 vars in 926 bytes} [Wed Jun 15 10:18:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:18:03.673775 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeaa676b-4e2c-46f3-a821-5b7c93091fe6 None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:03.674184 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeaa676b-4e2c-46f3-a821-5b7c93091fe6 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:03.674374 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeaa676b-4e2c-46f3-a821-5b7c93091fe6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:03.692015 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-eeaa676b-4e2c-46f3-a821-5b7c93091fe6 None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:18:03.693226 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-eeaa676b-4e2c-46f3-a821-5b7c93091fe6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:18:03.695229 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:18:03.771232 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 104/210] 10.222.0.49 () {58 vars in 1023 bytes} [Wed Jun 15 10:18:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 15 10:18:03.788439 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aad58ab-cf8b-4267-a578-5362eda87eea None None] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:18:03.805315 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=100 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:18:03.833643 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4854efe0-b684-4401-aee0-39189e0bc412 None None] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:03.833937 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4854efe0-b684-4401-aee0-39189e0bc412 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:03.834215 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4854efe0-b684-4401-aee0-39189e0bc412 None None] PATH_INFO: `/` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:03.835629 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 105/211] 10.222.0.49 () {54 vars in 926 bytes} [Wed Jun 15 10:18:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:18:03.842078 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c244bd81-b0e8-4d34-9054-5d25e8cdcd21 None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:03.842664 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c244bd81-b0e8-4d34-9054-5d25e8cdcd21 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:03.842949 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c244bd81-b0e8-4d34-9054-5d25e8cdcd21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:03.863971 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-c244bd81-b0e8-4d34-9054-5d25e8cdcd21 None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:18:03.865909 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-c244bd81-b0e8-4d34-9054-5d25e8cdcd21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:18:03.905193 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aad58ab-cf8b-4267-a578-5362eda87eea None None] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:18:03.938114 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 106/212] 10.222.0.49 () {58 vars in 1023 bytes} [Wed Jun 15 10:18:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 10:18:03.947920 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d4658f-1570-4a50-96d0-84b13b84bb48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:18:03.948784 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d4658f-1570-4a50-96d0-84b13b84bb48 None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:03.949072 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d4658f-1570-4a50-96d0-84b13b84bb48 None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:03.949351 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d4658f-1570-4a50-96d0-84b13b84bb48 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:03.960589 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d4658f-1570-4a50-96d0-84b13b84bb48 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 10:18:03.963516 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c244bd81-b0e8-4d34-9054-5d25e8cdcd21 None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:18:03.970964 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application [None req-31d4658f-1570-4a50-96d0-84b13b84bb48 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 15 10:18:03.970964 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:18:03.970964 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:18:03.970964 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:18:03.970964 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:18:03.970964 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:18:03.970964 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:18:03.970964 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:18:03.970964 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:18:03.970964 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:18:03.970964 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:18:03.970964 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:18:03.970964 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 10:18:03.970964 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 10:18:03.970964 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 10:18:03.970964 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 10:18:03.971887 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 10:18:03.971887 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 10:18:03.971887 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 10:18:03.971887 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 10:18:03.971887 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:18:03.971887 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:18:03.971887 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:18:03.971887 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:18:03.971887 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:18:03.971887 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:18:03.971887 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:18:03.971887 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application with Lock( Jun 15 10:18:03.971887 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:18:03.971887 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:18:03.971887 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:18:03.971887 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:18:03.972663 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:18:03.972663 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:18:03.972663 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:18:03.972663 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:18:03.972663 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 10:18:03.972663 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 10:18:03.972663 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 10:18:03.972663 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 10:18:03.972663 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 10:18:03.972663 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 10:18:03.972663 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 15 10:18:03.972663 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application Jun 15 10:18:03.973742 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 107/213] 10.222.0.49 () {56 vars in 1099 bytes} [Wed Jun 15 10:18:03 2022] GET /identity/v3/roles/member => generated 84 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:18:03.978364 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d4658f-1570-4a50-96d0-84b13b84bb48 None admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:18:04.048218 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=100 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:18:04.054541 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:18:04.086643 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d4658f-1570-4a50-96d0-84b13b84bb48 None admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:18:04.099714 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c244bd81-b0e8-4d34-9054-5d25e8cdcd21 None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:18:04.130669 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917f86c4-580d-4364-a612-44566198ef72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:18:04.132172 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917f86c4-580d-4364-a612-44566198ef72 None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:04.132630 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917f86c4-580d-4364-a612-44566198ef72 None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:04.133054 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917f86c4-580d-4364-a612-44566198ef72 None admin] PATH_INFO: `/v3/roles` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:04.134419 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917f86c4-580d-4364-a612-44566198ef72 None admin] RBAC: Adding query filter params (name=member) {{(pid=80119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:18:04.135037 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917f86c4-580d-4364-a612-44566198ef72 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:18:04.136970 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917f86c4-580d-4364-a612-44566198ef72 None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:18:04.142493 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149aa68c-90b9-4b6f-8e6b-0a405a4b5800 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:18:04.143456 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149aa68c-90b9-4b6f-8e6b-0a405a4b5800 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:04.143841 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149aa68c-90b9-4b6f-8e6b-0a405a4b5800 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:04.144215 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149aa68c-90b9-4b6f-8e6b-0a405a4b5800 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:04.145513 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-149aa68c-90b9-4b6f-8e6b-0a405a4b5800 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:18:04.147052 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-149aa68c-90b9-4b6f-8e6b-0a405a4b5800 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:18:04.148923 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 108/214] 10.222.0.49 () {56 vars in 1101 bytes} [Wed Jun 15 10:18:03 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 10:18:04.155205 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 107/215] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Jun 15 10:18:03 2022] GET /identity/v3/domains/default => generated 199 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 10:18:04.156679 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917f86c4-580d-4364-a612-44566198ef72 None admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:18:04.157886 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149aa68c-90b9-4b6f-8e6b-0a405a4b5800 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:18:04.273435 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149aa68c-90b9-4b6f-8e6b-0a405a4b5800 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:18:04.281477 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917f86c4-580d-4364-a612-44566198ef72 None admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:18:04.309205 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=100 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:18:04.319230 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4516584b-4501-47ee-b473-1a04f57a9278 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:18:04.320120 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4516584b-4501-47ee-b473-1a04f57a9278 None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:04.320426 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4516584b-4501-47ee-b473-1a04f57a9278 None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:04.320683 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4516584b-4501-47ee-b473-1a04f57a9278 None admin] PATH_INFO: `/v3/users/4bef473873954d20ad88a891d1c0f60e` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:04.325190 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56c9ea2-1158-444b-8398-e2cc7658d4a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:18:04.326773 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56c9ea2-1158-444b-8398-e2cc7658d4a0 None admin] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:04.327299 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56c9ea2-1158-444b-8398-e2cc7658d4a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:04.327779 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56c9ea2-1158-444b-8398-e2cc7658d4a0 None admin] PATH_INFO: `/v3/users` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:04.328695 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e56c9ea2-1158-444b-8398-e2cc7658d4a0 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:18:04.330886 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e56c9ea2-1158-444b-8398-e2cc7658d4a0 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:18:04.339430 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:18:04.341556 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4516584b-4501-47ee-b473-1a04f57a9278 None admin] RBAC: Authorizing `identity:get_user(user_id=4bef473873954d20ad88a891d1c0f60e)` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:18:04.344594 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4516584b-4501-47ee-b473-1a04f57a9278 None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:18:04.348044 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 109/216] 10.222.0.49 () {56 vars in 1177 bytes} [Wed Jun 15 10:18:04 2022] GET /identity/v3/users/4bef473873954d20ad88a891d1c0f60e => generated 297 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:18:04.353689 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4516584b-4501-47ee-b473-1a04f57a9278 None admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:18:04.362965 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 108/217] 10.222.0.49 () {60 vars in 1132 bytes} [Wed Jun 15 10:18:04 2022] POST /identity/v3/users => generated 297 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 10:18:04.473880 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4516584b-4501-47ee-b473-1a04f57a9278 None admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:18:04.517423 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67909079-05e7-4852-b7c9-d6dee884d0b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:18:04.518417 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67909079-05e7-4852-b7c9-d6dee884d0b8 None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:04.518878 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67909079-05e7-4852-b7c9-d6dee884d0b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:04.519389 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67909079-05e7-4852-b7c9-d6dee884d0b8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:04.557849 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e8d5b5-866a-4402-a25d-177fb48b6a2f None None] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:04.558144 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e8d5b5-866a-4402-a25d-177fb48b6a2f None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:04.558354 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e8d5b5-866a-4402-a25d-177fb48b6a2f None None] PATH_INFO: `/` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:04.559789 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 109/218] 10.222.0.49 () {54 vars in 926 bytes} [Wed Jun 15 10:18:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 15 10:18:04.565730 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d5191d-6b23-4975-beb9-69e08cac4fc0 None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:04.566008 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d5191d-6b23-4975-beb9-69e08cac4fc0 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:04.566290 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d5191d-6b23-4975-beb9-69e08cac4fc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:04.584529 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-e1d5191d-6b23-4975-beb9-69e08cac4fc0 None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:18:04.586030 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=100 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:18:04.586166 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-e1d5191d-6b23-4975-beb9-69e08cac4fc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:18:04.600031 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67909079-05e7-4852-b7c9-d6dee884d0b8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:18:04.601589 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67909079-05e7-4852-b7c9-d6dee884d0b8 None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:18:04.665159 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 110/219] 10.222.0.49 () {58 vars in 1023 bytes} [Wed Jun 15 10:18:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 15 10:18:04.686479 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39bf28f-3b1e-4074-ba2a-9d63bd51c044 None None] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:04.687076 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39bf28f-3b1e-4074-ba2a-9d63bd51c044 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:04.687076 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39bf28f-3b1e-4074-ba2a-9d63bd51c044 None None] PATH_INFO: `/` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:04.688447 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 111/220] 10.222.0.49 () {54 vars in 926 bytes} [Wed Jun 15 10:18:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 15 10:18:04.690003 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39bf28f-3b1e-4074-ba2a-9d63bd51c044 None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:18:04.724857 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 110/221] 10.222.0.49 () {58 vars in 1282 bytes} [Wed Jun 15 10:18:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 15 10:18:04.726180 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e451ff24-a036-4469-b721-0d88661ab308 None None] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:04.726533 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e451ff24-a036-4469-b721-0d88661ab308 None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:04.726827 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e451ff24-a036-4469-b721-0d88661ab308 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:04.748427 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.auth.core [None req-e451ff24-a036-4469-b721-0d88661ab308 None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:18:04.749681 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-e451ff24-a036-4469-b721-0d88661ab308 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:18:04.823896 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39bf28f-3b1e-4074-ba2a-9d63bd51c044 None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:18:04.837587 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 111/222] 10.222.0.49 () {58 vars in 1023 bytes} [Wed Jun 15 10:18:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 10:18:04.838791 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e451ff24-a036-4469-b721-0d88661ab308 None None] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:18:04.848089 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:18:04.859796 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=100 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:18:04.863340 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309b82aa-884e-48cc-92f2-85ac3a1a2941 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:18:04.865084 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309b82aa-884e-48cc-92f2-85ac3a1a2941 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:04.865784 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309b82aa-884e-48cc-92f2-85ac3a1a2941 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:04.866077 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309b82aa-884e-48cc-92f2-85ac3a1a2941 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:04.883985 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-309b82aa-884e-48cc-92f2-85ac3a1a2941 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 10:18:04.894604 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application [None req-309b82aa-884e-48cc-92f2-85ac3a1a2941 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 10:18:04.894604 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:18:04.894604 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:18:04.894604 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:18:04.894604 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:18:04.894604 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:18:04.894604 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:18:04.894604 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:18:04.894604 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:18:04.894604 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:18:04.894604 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:18:04.894604 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:18:04.894604 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 10:18:04.894604 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 10:18:04.894604 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 10:18:04.894604 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 10:18:04.895534 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 10:18:04.895534 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 10:18:04.895534 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 10:18:04.895534 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 10:18:04.895534 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:18:04.895534 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:18:04.895534 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:18:04.895534 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:18:04.895534 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:18:04.895534 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:18:04.895534 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:18:04.895534 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application with Lock( Jun 15 10:18:04.895534 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:18:04.895534 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:18:04.895534 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:18:04.895534 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:18:04.896342 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:18:04.896342 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:18:04.896342 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:18:04.896342 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:18:04.896342 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 10:18:04.896342 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 10:18:04.896342 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 10:18:04.896342 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 10:18:04.896342 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 10:18:04.896342 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 10:18:04.896342 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 10:18:04.896342 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Jun 15 10:18:04.896342 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 112/223] 10.222.0.49 () {56 vars in 1096 bytes} [Wed Jun 15 10:18:04 2022] GET /identity/v3/roles/admin => generated 83 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:18:04.897533 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309b82aa-884e-48cc-92f2-85ac3a1a2941 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:18:04.961995 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e451ff24-a036-4469-b721-0d88661ab308 None None] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:18:05.000929 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeda12ab-2596-4bdf-9f45-2c93ad644b93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:18:05.001858 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeda12ab-2596-4bdf-9f45-2c93ad644b93 None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:05.002141 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeda12ab-2596-4bdf-9f45-2c93ad644b93 None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:05.002468 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeda12ab-2596-4bdf-9f45-2c93ad644b93 None admin] PATH_INFO: `/v3/projects/df1f045b58c24b4dba0964eff7d2e540` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:05.003854 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeda12ab-2596-4bdf-9f45-2c93ad644b93 None admin] RBAC: Authorizing `identity:get_project(project_id=df1f045b58c24b4dba0964eff7d2e540)` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:18:05.005378 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeda12ab-2596-4bdf-9f45-2c93ad644b93 None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:18:05.006685 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309b82aa-884e-48cc-92f2-85ac3a1a2941 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:18:05.007160 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 112/224] 10.222.0.49 () {56 vars in 1186 bytes} [Wed Jun 15 10:18:04 2022] GET /identity/v3/projects/df1f045b58c24b4dba0964eff7d2e540 => generated 302 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:18:05.007934 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeda12ab-2596-4bdf-9f45-2c93ad644b93 None admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:18:05.042571 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f01045-ea9e-4bb4-95df-86c32e8e65d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:18:05.043429 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f01045-ea9e-4bb4-95df-86c32e8e65d9 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:05.043716 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f01045-ea9e-4bb4-95df-86c32e8e65d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:05.043995 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f01045-ea9e-4bb4-95df-86c32e8e65d9 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:05.055436 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f01045-ea9e-4bb4-95df-86c32e8e65d9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 10:18:05.063815 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application [None req-c0f01045-ea9e-4bb4-95df-86c32e8e65d9 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 15 10:18:05.063815 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:18:05.063815 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:18:05.063815 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:18:05.063815 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:18:05.063815 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:18:05.063815 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:18:05.063815 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:18:05.063815 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:18:05.063815 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:18:05.063815 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:18:05.063815 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:18:05.063815 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 10:18:05.063815 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 10:18:05.063815 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 10:18:05.063815 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 10:18:05.065925 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 10:18:05.065925 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 10:18:05.065925 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 10:18:05.065925 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 10:18:05.065925 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:18:05.065925 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:18:05.065925 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:18:05.065925 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:18:05.065925 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:18:05.065925 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:18:05.065925 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:18:05.065925 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application with Lock( Jun 15 10:18:05.065925 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:18:05.065925 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:18:05.065925 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:18:05.065925 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:18:05.067985 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:18:05.067985 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:18:05.067985 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:18:05.067985 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:18:05.067985 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 10:18:05.067985 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 10:18:05.067985 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 10:18:05.067985 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 10:18:05.067985 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 10:18:05.067985 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 10:18:05.067985 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 15 10:18:05.067985 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Jun 15 10:18:05.067985 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 113/225] 10.222.0.49 () {56 vars in 1114 bytes} [Wed Jun 15 10:18:04 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:18:05.067985 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f01045-ea9e-4bb4-95df-86c32e8e65d9 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:18:05.101148 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=100 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:18:05.119566 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:18:05.120510 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeda12ab-2596-4bdf-9f45-2c93ad644b93 None admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:18:05.156731 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580aa1e3-eba0-40c0-a44d-7f7096d79806 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:18:05.157693 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580aa1e3-eba0-40c0-a44d-7f7096d79806 None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:05.158118 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580aa1e3-eba0-40c0-a44d-7f7096d79806 None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:05.158531 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580aa1e3-eba0-40c0-a44d-7f7096d79806 None admin] PATH_INFO: `/v3/roles` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:05.159440 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580aa1e3-eba0-40c0-a44d-7f7096d79806 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:18:05.159889 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580aa1e3-eba0-40c0-a44d-7f7096d79806 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:18:05.161443 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580aa1e3-eba0-40c0-a44d-7f7096d79806 None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:18:05.175253 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f01045-ea9e-4bb4-95df-86c32e8e65d9 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:18:05.175555 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 113/226] 10.222.0.49 () {56 vars in 1099 bytes} [Wed Jun 15 10:18:05 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:18:05.176619 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580aa1e3-eba0-40c0-a44d-7f7096d79806 None admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:18:05.209308 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62a80b6-7330-4dcc-b031-22718ffdefa1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:18:05.209981 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62a80b6-7330-4dcc-b031-22718ffdefa1 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:05.210329 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62a80b6-7330-4dcc-b031-22718ffdefa1 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:05.210612 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62a80b6-7330-4dcc-b031-22718ffdefa1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:05.211531 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c62a80b6-7330-4dcc-b031-22718ffdefa1 None admin] RBAC: Adding query filter params (role.id=6876e57cc51c448097ab109d35b2a3da, scope.project.id=df1f045b58c24b4dba0964eff7d2e540, user.id=4bef473873954d20ad88a891d1c0f60e) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:18:05.211898 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c62a80b6-7330-4dcc-b031-22718ffdefa1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:18:05.213372 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c62a80b6-7330-4dcc-b031-22718ffdefa1 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:18:05.220045 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 114/227] 10.222.0.49 () {56 vars in 1374 bytes} [Wed Jun 15 10:18:05 2022] GET /identity/v3/role_assignments?user.id=4bef473873954d20ad88a891d1c0f60e&scope.project.id=df1f045b58c24b4dba0964eff7d2e540&role.id=6876e57cc51c448097ab109d35b2a3da => generated 259 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:18:05.221062 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62a80b6-7330-4dcc-b031-22718ffdefa1 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:18:05.320605 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580aa1e3-eba0-40c0-a44d-7f7096d79806 None admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:18:05.384973 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62a80b6-7330-4dcc-b031-22718ffdefa1 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:18:05.392123 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71c1c7e-1442-4018-afaa-383903d43c74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:18:05.392825 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71c1c7e-1442-4018-afaa-383903d43c74 None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:05.393369 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:18:05.394308 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71c1c7e-1442-4018-afaa-383903d43c74 None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:05.394796 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71c1c7e-1442-4018-afaa-383903d43c74 None admin] PATH_INFO: `/v3/roles` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:05.395861 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71c1c7e-1442-4018-afaa-383903d43c74 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:18:05.396397 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71c1c7e-1442-4018-afaa-383903d43c74 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:18:05.398027 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71c1c7e-1442-4018-afaa-383903d43c74 None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:18:05.408979 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 114/228] 10.222.0.49 () {56 vars in 1111 bytes} [Wed Jun 15 10:18:05 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:18:05.420461 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71c1c7e-1442-4018-afaa-383903d43c74 None admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:18:05.430676 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420b77b3-c136-4bee-9561-9ee2b4bb9c07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:18:05.431598 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420b77b3-c136-4bee-9561-9ee2b4bb9c07 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:05.431887 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420b77b3-c136-4bee-9561-9ee2b4bb9c07 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:05.432169 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420b77b3-c136-4bee-9561-9ee2b4bb9c07 None admin] PATH_INFO: `/v3/users/c3f14bfbae9a41858def110ed6fe1d8e` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:05.433471 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420b77b3-c136-4bee-9561-9ee2b4bb9c07 None admin] RBAC: Authorizing `identity:get_user(user_id=c3f14bfbae9a41858def110ed6fe1d8e)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:18:05.439112 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420b77b3-c136-4bee-9561-9ee2b4bb9c07 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:18:05.439112 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 115/229] 10.222.0.49 () {56 vars in 1177 bytes} [Wed Jun 15 10:18:05 2022] GET /identity/v3/users/c3f14bfbae9a41858def110ed6fe1d8e => generated 283 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:18:05.442116 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420b77b3-c136-4bee-9561-9ee2b4bb9c07 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:18:05.443597 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=100 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:18:05.550837 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420b77b3-c136-4bee-9561-9ee2b4bb9c07 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:18:05.598860 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71c1c7e-1442-4018-afaa-383903d43c74 None admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:18:05.601705 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7644f048-84e0-4552-b070-b25b1474a432 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:18:05.602639 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7644f048-84e0-4552-b070-b25b1474a432 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:05.602929 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7644f048-84e0-4552-b070-b25b1474a432 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:05.603240 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7644f048-84e0-4552-b070-b25b1474a432 None admin] PATH_INFO: `/v3/projects/df1f045b58c24b4dba0964eff7d2e540` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:05.605315 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7644f048-84e0-4552-b070-b25b1474a432 None admin] RBAC: Authorizing `identity:get_project(project_id=df1f045b58c24b4dba0964eff7d2e540)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:18:05.606784 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7644f048-84e0-4552-b070-b25b1474a432 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:18:05.611251 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 116/230] 10.222.0.49 () {56 vars in 1186 bytes} [Wed Jun 15 10:18:05 2022] GET /identity/v3/projects/df1f045b58c24b4dba0964eff7d2e540 => generated 302 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:18:05.613918 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7644f048-84e0-4552-b070-b25b1474a432 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:18:05.644360 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da34cfb-7696-4221-858e-1a3bf9923b4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:18:05.645228 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da34cfb-7696-4221-858e-1a3bf9923b4b None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:05.645538 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da34cfb-7696-4221-858e-1a3bf9923b4b None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:05.646292 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da34cfb-7696-4221-858e-1a3bf9923b4b None admin] PATH_INFO: `/v3/users/c3f14bfbae9a41858def110ed6fe1d8e` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:05.647843 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da34cfb-7696-4221-858e-1a3bf9923b4b None admin] RBAC: Authorizing `identity:get_user(user_id=c3f14bfbae9a41858def110ed6fe1d8e)` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:18:05.649653 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da34cfb-7696-4221-858e-1a3bf9923b4b None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:18:05.652099 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 115/231] 10.222.0.49 () {56 vars in 1177 bytes} [Wed Jun 15 10:18:05 2022] GET /identity/v3/users/c3f14bfbae9a41858def110ed6fe1d8e => generated 283 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:18:05.658185 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da34cfb-7696-4221-858e-1a3bf9923b4b None admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:18:05.681045 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:18:05.741445 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7644f048-84e0-4552-b070-b25b1474a432 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:18:05.782034 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4452a117-dd12-4fa4-9c15-f2bccbc491c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:18:05.783352 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4452a117-dd12-4fa4-9c15-f2bccbc491c1 None admin] REQUEST_METHOD: `PUT` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:05.783838 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4452a117-dd12-4fa4-9c15-f2bccbc491c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:05.784312 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4452a117-dd12-4fa4-9c15-f2bccbc491c1 None admin] PATH_INFO: `/v3/projects/df1f045b58c24b4dba0964eff7d2e540/users/c3f14bfbae9a41858def110ed6fe1d8e/roles/8051174e7a7947d5b991ebd793c2e543` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:05.786614 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4452a117-dd12-4fa4-9c15-f2bccbc491c1 None admin] RBAC: Authorizing `identity:create_grant(project_id=df1f045b58c24b4dba0964eff7d2e540, user_id=c3f14bfbae9a41858def110ed6fe1d8e, role_id=8051174e7a7947d5b991ebd793c2e543)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:18:05.788293 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4452a117-dd12-4fa4-9c15-f2bccbc491c1 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:18:05.811192 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 117/232] 10.222.0.49 () {58 vars in 1439 bytes} [Wed Jun 15 10:18:05 2022] PUT /identity/v3/projects/df1f045b58c24b4dba0964eff7d2e540/users/c3f14bfbae9a41858def110ed6fe1d8e/roles/8051174e7a7947d5b991ebd793c2e543 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 10:18:05.846148 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=100 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:18:05.876603 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da34cfb-7696-4221-858e-1a3bf9923b4b None admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:18:05.917451 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c99595-88be-487e-9575-61d723c1b4e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:18:05.918470 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c99595-88be-487e-9575-61d723c1b4e6 None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:05.918848 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c99595-88be-487e-9575-61d723c1b4e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:05.919262 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c99595-88be-487e-9575-61d723c1b4e6 None admin] PATH_INFO: `/v3/projects/df1f045b58c24b4dba0964eff7d2e540` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:05.920659 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c99595-88be-487e-9575-61d723c1b4e6 None admin] RBAC: Authorizing `identity:get_project(project_id=df1f045b58c24b4dba0964eff7d2e540)` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:18:05.922155 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c99595-88be-487e-9575-61d723c1b4e6 None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:18:05.924388 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 116/233] 10.222.0.49 () {56 vars in 1186 bytes} [Wed Jun 15 10:18:05 2022] GET /identity/v3/projects/df1f045b58c24b4dba0964eff7d2e540 => generated 302 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:18:05.930192 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4452a117-dd12-4fa4-9c15-f2bccbc491c1 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:18:06.057272 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4452a117-dd12-4fa4-9c15-f2bccbc491c1 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:18:06.058765 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:18:06.095441 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b043d4a-3a1e-4db1-b9d0-bd962fe40aae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:18:06.096343 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b043d4a-3a1e-4db1-b9d0-bd962fe40aae None admin] REQUEST_METHOD: `PUT` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:06.096628 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b043d4a-3a1e-4db1-b9d0-bd962fe40aae None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:06.096904 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b043d4a-3a1e-4db1-b9d0-bd962fe40aae None admin] PATH_INFO: `/v3/projects/df1f045b58c24b4dba0964eff7d2e540/users/c3f14bfbae9a41858def110ed6fe1d8e/roles/4d2118569d074a288b37d120110c300d` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:06.099209 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b043d4a-3a1e-4db1-b9d0-bd962fe40aae None admin] RBAC: Authorizing `identity:create_grant(project_id=df1f045b58c24b4dba0964eff7d2e540, user_id=c3f14bfbae9a41858def110ed6fe1d8e, role_id=4d2118569d074a288b37d120110c300d)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:18:06.100664 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b043d4a-3a1e-4db1-b9d0-bd962fe40aae None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:18:06.123570 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 118/234] 10.222.0.49 () {58 vars in 1439 bytes} [Wed Jun 15 10:18:05 2022] PUT /identity/v3/projects/df1f045b58c24b4dba0964eff7d2e540/users/c3f14bfbae9a41858def110ed6fe1d8e/roles/4d2118569d074a288b37d120110c300d => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 10:18:07.024822 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff9071a-6502-49f4-ba24-687d0d5ca19c None None] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:07.025356 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff9071a-6502-49f4-ba24-687d0d5ca19c None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:07.025356 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff9071a-6502-49f4-ba24-687d0d5ca19c None None] PATH_INFO: `/` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:07.026740 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 117/235] 10.222.0.49 () {54 vars in 926 bytes} [Wed Jun 15 10:18:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:18:07.032592 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a2d6a7-4eb1-4340-82df-8da2445fa3f3 None None] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:07.034558 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f92cd55-d5aa-4cf0-9188-09fc593a2253 None None] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:07.035053 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f92cd55-d5aa-4cf0-9188-09fc593a2253 None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:07.035554 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f92cd55-d5aa-4cf0-9188-09fc593a2253 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:07.038105 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a2d6a7-4eb1-4340-82df-8da2445fa3f3 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:07.039720 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a2d6a7-4eb1-4340-82df-8da2445fa3f3 None None] PATH_INFO: `/` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:07.042918 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 119/236] 10.222.0.49 () {54 vars in 926 bytes} [Wed Jun 15 10:18:07 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:18:07.048957 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31628ce7-799a-488f-a8c1-8cd2fa8ed461 None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:07.054931 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31628ce7-799a-488f-a8c1-8cd2fa8ed461 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:07.055472 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31628ce7-799a-488f-a8c1-8cd2fa8ed461 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:07.072403 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.auth.core [None req-7f92cd55-d5aa-4cf0-9188-09fc593a2253 None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:18:07.073920 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-7f92cd55-d5aa-4cf0-9188-09fc593a2253 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:18:07.086383 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-31628ce7-799a-488f-a8c1-8cd2fa8ed461 None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:18:07.087847 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-31628ce7-799a-488f-a8c1-8cd2fa8ed461 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:18:07.202620 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 118/237] 10.222.0.49 () {58 vars in 1023 bytes} [Wed Jun 15 10:18:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 15 10:18:07.205423 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 120/238] 10.222.0.49 () {58 vars in 1023 bytes} [Wed Jun 15 10:18:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 10:18:07.230110 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f92cd55-d5aa-4cf0-9188-09fc593a2253 None None] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:18:07.238243 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31628ce7-799a-488f-a8c1-8cd2fa8ed461 None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:18:07.371538 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=100 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:18:07.391785 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:18:07.405756 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31628ce7-799a-488f-a8c1-8cd2fa8ed461 None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:18:07.411868 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f92cd55-d5aa-4cf0-9188-09fc593a2253 None None] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:18:07.469272 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc99c34-f5ca-4a06-ac1a-d29fabcb9483 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:18:07.470085 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc99c34-f5ca-4a06-ac1a-d29fabcb9483 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:07.470365 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc99c34-f5ca-4a06-ac1a-d29fabcb9483 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:07.470641 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc99c34-f5ca-4a06-ac1a-d29fabcb9483 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:07.473679 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9080ba88-9e51-4dac-8770-31289afeecf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:18:07.474471 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9080ba88-9e51-4dac-8770-31289afeecf7 None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:07.474751 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9080ba88-9e51-4dac-8770-31289afeecf7 None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:07.475021 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9080ba88-9e51-4dac-8770-31289afeecf7 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:07.476896 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9080ba88-9e51-4dac-8770-31289afeecf7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:18:07.478377 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9080ba88-9e51-4dac-8770-31289afeecf7 None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:18:07.480196 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 119/239] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Jun 15 10:18:07 2022] GET /identity/v3/domains/default => generated 199 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:18:07.485421 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9080ba88-9e51-4dac-8770-31289afeecf7 None admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:18:07.491645 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc99c34-f5ca-4a06-ac1a-d29fabcb9483 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 10:18:07.499865 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application [None req-5cc99c34-f5ca-4a06-ac1a-d29fabcb9483 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 15 10:18:07.499865 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:18:07.499865 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:18:07.499865 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:18:07.499865 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:18:07.499865 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:18:07.499865 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:18:07.499865 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:18:07.499865 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:18:07.499865 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:18:07.499865 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:18:07.499865 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:18:07.499865 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 10:18:07.499865 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 10:18:07.499865 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 10:18:07.499865 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 10:18:07.500740 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 10:18:07.500740 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 10:18:07.500740 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 10:18:07.500740 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 10:18:07.500740 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:18:07.500740 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:18:07.500740 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:18:07.500740 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:18:07.500740 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:18:07.500740 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:18:07.500740 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:18:07.500740 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application with Lock( Jun 15 10:18:07.500740 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:18:07.500740 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:18:07.500740 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:18:07.500740 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:18:07.501545 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:18:07.501545 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:18:07.501545 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:18:07.501545 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:18:07.501545 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 10:18:07.501545 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 10:18:07.501545 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 10:18:07.501545 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 10:18:07.501545 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 10:18:07.501545 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 10:18:07.501545 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 15 10:18:07.501545 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Jun 15 10:18:07.506006 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 121/240] 10.222.0.49 () {56 vars in 1099 bytes} [Wed Jun 15 10:18:07 2022] GET /identity/v3/roles/reader => generated 84 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:18:07.508106 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc99c34-f5ca-4a06-ac1a-d29fabcb9483 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:18:07.667655 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9080ba88-9e51-4dac-8770-31289afeecf7 None admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:18:07.685906 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc99c34-f5ca-4a06-ac1a-d29fabcb9483 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:18:07.734910 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63b7024-eb59-4ca7-8a76-3b94808b26cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:18:07.736595 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63b7024-eb59-4ca7-8a76-3b94808b26cb None admin] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:07.737913 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63b7024-eb59-4ca7-8a76-3b94808b26cb None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:07.738336 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63b7024-eb59-4ca7-8a76-3b94808b26cb None admin] PATH_INFO: `/v3/users` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:07.740481 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63b7024-eb59-4ca7-8a76-3b94808b26cb None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:18:07.742186 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63b7024-eb59-4ca7-8a76-3b94808b26cb None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:18:07.760127 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2311104d-2f8e-46b2-9b04-4fbc7466d577 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:18:07.761084 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2311104d-2f8e-46b2-9b04-4fbc7466d577 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:07.761511 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2311104d-2f8e-46b2-9b04-4fbc7466d577 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:07.761924 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2311104d-2f8e-46b2-9b04-4fbc7466d577 None admin] PATH_INFO: `/v3/roles` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:07.762870 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2311104d-2f8e-46b2-9b04-4fbc7466d577 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:18:07.763400 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2311104d-2f8e-46b2-9b04-4fbc7466d577 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:18:07.765010 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2311104d-2f8e-46b2-9b04-4fbc7466d577 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:18:07.775525 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 122/241] 10.222.0.49 () {56 vars in 1101 bytes} [Wed Jun 15 10:18:07 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:18:07.782750 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2311104d-2f8e-46b2-9b04-4fbc7466d577 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:18:07.785063 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:18:07.792683 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 120/242] 10.222.0.49 () {60 vars in 1132 bytes} [Wed Jun 15 10:18:07 2022] POST /identity/v3/users => generated 293 bytes in 307 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 10:18:07.814467 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:18:07.854373 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36699c6c-80a9-47b7-8eb5-087c86b00ad6 None None] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:07.855116 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36699c6c-80a9-47b7-8eb5-087c86b00ad6 None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:07.855655 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36699c6c-80a9-47b7-8eb5-087c86b00ad6 None None] PATH_INFO: `/` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:07.858389 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 121/243] 10.222.0.49 () {54 vars in 926 bytes} [Wed Jun 15 10:18:07 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:18:07.864706 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb8d2b4-3590-4fe8-a227-4062467feced None None] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:07.865331 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb8d2b4-3590-4fe8-a227-4062467feced None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:07.865810 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb8d2b4-3590-4fe8-a227-4062467feced None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:07.899291 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.auth.core [None req-5fb8d2b4-3590-4fe8-a227-4062467feced None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:18:07.900543 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-5fb8d2b4-3590-4fe8-a227-4062467feced None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:18:07.942199 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2311104d-2f8e-46b2-9b04-4fbc7466d577 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:18:07.996220 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c079d58-c6e1-49e4-8e9b-9be21726fd13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:18:07.997088 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c079d58-c6e1-49e4-8e9b-9be21726fd13 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:07.997373 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c079d58-c6e1-49e4-8e9b-9be21726fd13 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:07.998300 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c079d58-c6e1-49e4-8e9b-9be21726fd13 None admin] PATH_INFO: `/v3/users/b142a76f3f994d00b76f1eecb82a7541` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:08.013473 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c079d58-c6e1-49e4-8e9b-9be21726fd13 None admin] RBAC: Authorizing `identity:get_user(user_id=b142a76f3f994d00b76f1eecb82a7541)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:18:08.014939 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c079d58-c6e1-49e4-8e9b-9be21726fd13 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:18:08.017370 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 123/244] 10.222.0.49 () {56 vars in 1177 bytes} [Wed Jun 15 10:18:07 2022] GET /identity/v3/users/b142a76f3f994d00b76f1eecb82a7541 => generated 297 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:18:08.018387 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 122/245] 10.222.0.49 () {58 vars in 1023 bytes} [Wed Jun 15 10:18:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 15 10:18:08.022807 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c079d58-c6e1-49e4-8e9b-9be21726fd13 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:18:08.043938 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb8d2b4-3590-4fe8-a227-4062467feced None None] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:18:08.099853 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:18:08.133898 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c079d58-c6e1-49e4-8e9b-9be21726fd13 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:18:08.165209 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb8d2b4-3590-4fe8-a227-4062467feced None None] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:18:08.171432 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6411ce6b-1df0-436e-94f5-9558a8564c14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:18:08.172095 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6411ce6b-1df0-436e-94f5-9558a8564c14 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:08.172382 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6411ce6b-1df0-436e-94f5-9558a8564c14 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:08.172659 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6411ce6b-1df0-436e-94f5-9558a8564c14 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:08.196568 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=100 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:18:08.207152 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309ebe33-a19b-4858-b956-77ef0e70e911 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:18:08.208362 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309ebe33-a19b-4858-b956-77ef0e70e911 None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:08.208906 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309ebe33-a19b-4858-b956-77ef0e70e911 None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:08.209388 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309ebe33-a19b-4858-b956-77ef0e70e911 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:08.221898 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-309ebe33-a19b-4858-b956-77ef0e70e911 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 10:18:08.230609 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application [None req-309ebe33-a19b-4858-b956-77ef0e70e911 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 15 10:18:08.230609 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:18:08.230609 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:18:08.230609 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:18:08.230609 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:18:08.230609 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:18:08.230609 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:18:08.230609 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:18:08.230609 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:18:08.230609 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:18:08.230609 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:18:08.230609 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:18:08.230609 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 10:18:08.230609 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 10:18:08.230609 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 10:18:08.230609 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 10:18:08.231494 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 10:18:08.231494 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 10:18:08.231494 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 10:18:08.231494 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 10:18:08.231494 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:18:08.231494 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:18:08.231494 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:18:08.231494 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:18:08.231494 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:18:08.231494 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:18:08.231494 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:18:08.231494 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application with Lock( Jun 15 10:18:08.231494 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:18:08.231494 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:18:08.231494 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:18:08.231494 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:18:08.232995 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:18:08.232995 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:18:08.232995 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:18:08.232995 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:18:08.232995 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 10:18:08.232995 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 10:18:08.232995 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 10:18:08.232995 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 10:18:08.232995 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 10:18:08.232995 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 10:18:08.232995 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 15 10:18:08.232995 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application Jun 15 10:18:08.235595 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 123/246] 10.222.0.49 () {56 vars in 1099 bytes} [Wed Jun 15 10:18:08 2022] GET /identity/v3/roles/member => generated 84 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 10:18:08.241756 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309ebe33-a19b-4858-b956-77ef0e70e911 None admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:18:08.244039 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6411ce6b-1df0-436e-94f5-9558a8564c14 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:18:08.245565 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6411ce6b-1df0-436e-94f5-9558a8564c14 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:18:08.320747 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:18:08.363215 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 124/247] 10.222.0.49 () {58 vars in 1282 bytes} [Wed Jun 15 10:18:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 15 10:18:08.365248 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef00788a-baae-4563-95ec-3a25fc608cb5 None None] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:08.365516 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef00788a-baae-4563-95ec-3a25fc608cb5 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:08.365795 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef00788a-baae-4563-95ec-3a25fc608cb5 None None] PATH_INFO: `/` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:08.367299 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 125/248] 10.222.0.49 () {54 vars in 926 bytes} [Wed Jun 15 10:18:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:18:08.372938 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef00788a-baae-4563-95ec-3a25fc608cb5 None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:18:08.374256 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309ebe33-a19b-4858-b956-77ef0e70e911 None admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:18:08.416611 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa05410-b4f8-48ec-853e-af48e1100a60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:18:08.417302 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa05410-b4f8-48ec-853e-af48e1100a60 None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:08.417589 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa05410-b4f8-48ec-853e-af48e1100a60 None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:08.417870 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa05410-b4f8-48ec-853e-af48e1100a60 None admin] PATH_INFO: `/v3/roles` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:08.418632 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa05410-b4f8-48ec-853e-af48e1100a60 None admin] RBAC: Adding query filter params (name=member) {{(pid=80119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:18:08.418941 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa05410-b4f8-48ec-853e-af48e1100a60 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:18:08.420428 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa05410-b4f8-48ec-853e-af48e1100a60 None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:18:08.429914 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 124/249] 10.222.0.49 () {56 vars in 1101 bytes} [Wed Jun 15 10:18:08 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 10:18:08.431971 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1847b458-12ee-4226-913d-b3d010be7004 None None] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:08.432469 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1847b458-12ee-4226-913d-b3d010be7004 None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:08.432962 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1847b458-12ee-4226-913d-b3d010be7004 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:08.453136 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.auth.core [None req-1847b458-12ee-4226-913d-b3d010be7004 None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:18:08.454659 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-1847b458-12ee-4226-913d-b3d010be7004 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:18:08.514612 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=100 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:18:08.528041 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef00788a-baae-4563-95ec-3a25fc608cb5 None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:18:08.551216 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 125/250] 10.222.0.49 () {58 vars in 1023 bytes} [Wed Jun 15 10:18:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 10:18:08.553230 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1847b458-12ee-4226-913d-b3d010be7004 None None] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:18:08.590553 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d21aee-ea9d-42b8-92c4-57a8931e9060 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:18:08.592902 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d21aee-ea9d-42b8-92c4-57a8931e9060 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:08.593195 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d21aee-ea9d-42b8-92c4-57a8931e9060 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:08.593472 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d21aee-ea9d-42b8-92c4-57a8931e9060 None admin] PATH_INFO: `/v3/projects/df1f045b58c24b4dba0964eff7d2e540` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:08.597632 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d21aee-ea9d-42b8-92c4-57a8931e9060 None admin] RBAC: Authorizing `identity:get_project(project_id=df1f045b58c24b4dba0964eff7d2e540)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:18:08.599124 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d21aee-ea9d-42b8-92c4-57a8931e9060 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:18:08.601926 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 126/251] 10.222.0.49 () {56 vars in 1186 bytes} [Wed Jun 15 10:18:08 2022] GET /identity/v3/projects/df1f045b58c24b4dba0964eff7d2e540 => generated 302 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:18:08.602705 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d21aee-ea9d-42b8-92c4-57a8931e9060 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:18:08.681824 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:18:08.734542 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1847b458-12ee-4226-913d-b3d010be7004 None None] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:18:08.787168 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d21aee-ea9d-42b8-92c4-57a8931e9060 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:18:08.810889 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bdd8087-a126-414f-8439-ea530197f524 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:18:08.811817 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdd8087-a126-414f-8439-ea530197f524 None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:08.812098 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdd8087-a126-414f-8439-ea530197f524 None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:08.812425 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdd8087-a126-414f-8439-ea530197f524 None admin] PATH_INFO: `/v3/users/4bef473873954d20ad88a891d1c0f60e` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:08.816903 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bdd8087-a126-414f-8439-ea530197f524 None admin] RBAC: Authorizing `identity:get_user(user_id=4bef473873954d20ad88a891d1c0f60e)` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:18:08.818395 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bdd8087-a126-414f-8439-ea530197f524 None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:18:08.820261 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 126/252] 10.222.0.49 () {56 vars in 1177 bytes} [Wed Jun 15 10:18:08 2022] GET /identity/v3/users/4bef473873954d20ad88a891d1c0f60e => generated 297 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:18:08.821000 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bdd8087-a126-414f-8439-ea530197f524 None admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:18:08.838813 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca0a90c-f911-4ce4-a6e5-957a74d3eb90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:18:08.839656 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca0a90c-f911-4ce4-a6e5-957a74d3eb90 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:08.840032 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca0a90c-f911-4ce4-a6e5-957a74d3eb90 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:08.840314 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca0a90c-f911-4ce4-a6e5-957a74d3eb90 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:08.861011 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca0a90c-f911-4ce4-a6e5-957a74d3eb90 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 10:18:08.876100 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=100 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:18:08.878286 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application [None req-eca0a90c-f911-4ce4-a6e5-957a74d3eb90 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 10:18:08.878286 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:18:08.878286 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:18:08.878286 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:18:08.878286 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:18:08.878286 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:18:08.878286 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:18:08.878286 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:18:08.878286 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:18:08.878286 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:18:08.878286 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:18:08.878286 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:18:08.878286 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 10:18:08.878286 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 10:18:08.878286 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 10:18:08.878286 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 10:18:08.879196 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 10:18:08.879196 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 10:18:08.879196 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 10:18:08.879196 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 10:18:08.879196 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:18:08.879196 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:18:08.879196 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:18:08.879196 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:18:08.879196 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:18:08.879196 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:18:08.879196 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:18:08.879196 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application with Lock( Jun 15 10:18:08.879196 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:18:08.879196 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:18:08.879196 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:18:08.879196 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:18:08.880041 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:18:08.880041 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:18:08.880041 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:18:08.880041 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:18:08.880041 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 10:18:08.880041 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 10:18:08.880041 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 10:18:08.880041 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 10:18:08.880041 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 10:18:08.880041 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 10:18:08.880041 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 10:18:08.880041 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Jun 15 10:18:08.880041 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 127/253] 10.222.0.49 () {56 vars in 1096 bytes} [Wed Jun 15 10:18:08 2022] GET /identity/v3/roles/admin => generated 83 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:18:08.880696 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca0a90c-f911-4ce4-a6e5-957a74d3eb90 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:18:08.934256 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bdd8087-a126-414f-8439-ea530197f524 None admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:18:08.969326 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ceb6b35-76f7-4e8b-abbe-2b482cdaaa35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:18:08.969984 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ceb6b35-76f7-4e8b-abbe-2b482cdaaa35 None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:08.970265 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ceb6b35-76f7-4e8b-abbe-2b482cdaaa35 None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:08.970536 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ceb6b35-76f7-4e8b-abbe-2b482cdaaa35 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:08.971391 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ceb6b35-76f7-4e8b-abbe-2b482cdaaa35 None admin] RBAC: Adding query filter params (role.id=301a3c8a23ba40edbc35df1da8af0de9, scope.project.id=df1f045b58c24b4dba0964eff7d2e540, user.id=b142a76f3f994d00b76f1eecb82a7541) {{(pid=80119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:18:08.971700 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ceb6b35-76f7-4e8b-abbe-2b482cdaaa35 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:18:08.973143 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ceb6b35-76f7-4e8b-abbe-2b482cdaaa35 None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:18:08.979010 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 127/254] 10.222.0.49 () {56 vars in 1374 bytes} [Wed Jun 15 10:18:08 2022] GET /identity/v3/role_assignments?user.id=b142a76f3f994d00b76f1eecb82a7541&scope.project.id=df1f045b58c24b4dba0964eff7d2e540&role.id=301a3c8a23ba40edbc35df1da8af0de9 => generated 259 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:18:08.979748 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ceb6b35-76f7-4e8b-abbe-2b482cdaaa35 None admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:18:09.093172 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ceb6b35-76f7-4e8b-abbe-2b482cdaaa35 None admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:18:09.116322 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=100 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:18:09.132525 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4081974-60f1-4063-9130-460681bf1077 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:18:09.133751 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4081974-60f1-4063-9130-460681bf1077 None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:09.134036 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4081974-60f1-4063-9130-460681bf1077 None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:09.134311 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4081974-60f1-4063-9130-460681bf1077 None admin] PATH_INFO: `/v3/roles` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:09.135074 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4081974-60f1-4063-9130-460681bf1077 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:18:09.135413 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4081974-60f1-4063-9130-460681bf1077 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:18:09.137174 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4081974-60f1-4063-9130-460681bf1077 None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:18:09.164132 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:18:09.165500 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 128/255] 10.222.0.49 () {56 vars in 1099 bytes} [Wed Jun 15 10:18:08 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:18:09.166966 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c90239-648a-43ca-ad6a-af054f61db78 None None] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:09.167315 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c90239-648a-43ca-ad6a-af054f61db78 None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:09.167643 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c90239-648a-43ca-ad6a-af054f61db78 None None] PATH_INFO: `/` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:09.168065 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca0a90c-f911-4ce4-a6e5-957a74d3eb90 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:18:09.169169 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 129/256] 10.222.0.49 () {54 vars in 926 bytes} [Wed Jun 15 10:18:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:18:09.180174 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c90239-648a-43ca-ad6a-af054f61db78 None None] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:18:09.210686 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9ac363-d758-4bad-9a18-2d0b1db69034 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:18:09.211881 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9ac363-d758-4bad-9a18-2d0b1db69034 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:09.212170 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9ac363-d758-4bad-9a18-2d0b1db69034 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:09.212468 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9ac363-d758-4bad-9a18-2d0b1db69034 None admin] PATH_INFO: `/v3/projects/df1f045b58c24b4dba0964eff7d2e540` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:09.214210 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b9ac363-d758-4bad-9a18-2d0b1db69034 None admin] RBAC: Authorizing `identity:get_project(project_id=df1f045b58c24b4dba0964eff7d2e540)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:18:09.215738 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b9ac363-d758-4bad-9a18-2d0b1db69034 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:18:09.218062 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 128/257] 10.222.0.49 () {56 vars in 1186 bytes} [Wed Jun 15 10:18:08 2022] GET /identity/v3/projects/df1f045b58c24b4dba0964eff7d2e540 => generated 302 bytes in 337 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:18:09.219300 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b33f22-b911-462f-a22c-2b208ec2ddc8 None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:09.219587 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b33f22-b911-462f-a22c-2b208ec2ddc8 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:09.219860 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b33f22-b911-462f-a22c-2b208ec2ddc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:09.241933 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-43b33f22-b911-462f-a22c-2b208ec2ddc8 None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:18:09.243722 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-43b33f22-b911-462f-a22c-2b208ec2ddc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:18:09.324918 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 129/258] 10.222.0.49 () {58 vars in 1023 bytes} [Wed Jun 15 10:18:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 10:18:09.325864 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b33f22-b911-462f-a22c-2b208ec2ddc8 None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:18:09.360856 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c90239-648a-43ca-ad6a-af054f61db78 None None] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:18:09.414627 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b7490b-2b74-4eda-9d04-0358c94d48d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:18:09.415911 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b7490b-2b74-4eda-9d04-0358c94d48d3 None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:09.416402 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b7490b-2b74-4eda-9d04-0358c94d48d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:09.416879 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b7490b-2b74-4eda-9d04-0358c94d48d3 None admin] PATH_INFO: `/v3/users/c3f14bfbae9a41858def110ed6fe1d8e` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:09.418355 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b7490b-2b74-4eda-9d04-0358c94d48d3 None admin] RBAC: Authorizing `identity:get_user(user_id=c3f14bfbae9a41858def110ed6fe1d8e)` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:18:09.420611 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b7490b-2b74-4eda-9d04-0358c94d48d3 None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:18:09.425158 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:18:09.429659 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b33f22-b911-462f-a22c-2b208ec2ddc8 None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:18:09.431487 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 130/259] 10.222.0.49 () {56 vars in 1177 bytes} [Wed Jun 15 10:18:09 2022] GET /identity/v3/users/c3f14bfbae9a41858def110ed6fe1d8e => generated 283 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:18:09.432284 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b7490b-2b74-4eda-9d04-0358c94d48d3 None admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:18:09.467468 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392d0e0f-7e8a-4dc0-811d-656c474f6e9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:18:09.468660 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392d0e0f-7e8a-4dc0-811d-656c474f6e9f None admin] REQUEST_METHOD: `PUT` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:09.469116 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392d0e0f-7e8a-4dc0-811d-656c474f6e9f None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:09.469562 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392d0e0f-7e8a-4dc0-811d-656c474f6e9f None admin] PATH_INFO: `/v3/projects/df1f045b58c24b4dba0964eff7d2e540/users/4bef473873954d20ad88a891d1c0f60e/roles/6876e57cc51c448097ab109d35b2a3da` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:09.474854 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-392d0e0f-7e8a-4dc0-811d-656c474f6e9f None admin] RBAC: Authorizing `identity:create_grant(project_id=df1f045b58c24b4dba0964eff7d2e540, user_id=4bef473873954d20ad88a891d1c0f60e, role_id=6876e57cc51c448097ab109d35b2a3da)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:18:09.476679 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-392d0e0f-7e8a-4dc0-811d-656c474f6e9f None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:18:09.505554 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 130/260] 10.222.0.49 () {58 vars in 1439 bytes} [Wed Jun 15 10:18:09 2022] PUT /identity/v3/projects/df1f045b58c24b4dba0964eff7d2e540/users/4bef473873954d20ad88a891d1c0f60e/roles/6876e57cc51c448097ab109d35b2a3da => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:18:09.506806 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392d0e0f-7e8a-4dc0-811d-656c474f6e9f None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:18:09.515955 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=100 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:18:09.628781 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b7490b-2b74-4eda-9d04-0358c94d48d3 None admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:18:09.689152 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392d0e0f-7e8a-4dc0-811d-656c474f6e9f None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:18:09.703819 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e757e1ba-b97d-4733-8b0b-d2d079627861 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:18:09.705033 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e757e1ba-b97d-4733-8b0b-d2d079627861 None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:09.705529 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e757e1ba-b97d-4733-8b0b-d2d079627861 None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:09.706012 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e757e1ba-b97d-4733-8b0b-d2d079627861 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:09.732672 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e757e1ba-b97d-4733-8b0b-d2d079627861 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 10:18:09.747114 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application [None req-e757e1ba-b97d-4733-8b0b-d2d079627861 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 15 10:18:09.747114 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:18:09.747114 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:18:09.747114 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:18:09.747114 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:18:09.747114 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:18:09.747114 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:18:09.747114 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:18:09.747114 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:18:09.747114 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:18:09.747114 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:18:09.747114 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:18:09.747114 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 10:18:09.747114 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 10:18:09.747114 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 10:18:09.747114 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 10:18:09.748217 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 10:18:09.748217 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 10:18:09.748217 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 10:18:09.748217 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 10:18:09.748217 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:18:09.748217 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:18:09.748217 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:18:09.748217 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:18:09.748217 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:18:09.748217 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:18:09.748217 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:18:09.748217 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application with Lock( Jun 15 10:18:09.748217 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:18:09.748217 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:18:09.748217 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:18:09.748217 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:18:09.749014 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:18:09.749014 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:18:09.749014 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:18:09.749014 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:18:09.749014 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 10:18:09.749014 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 10:18:09.749014 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 10:18:09.749014 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 10:18:09.749014 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 10:18:09.749014 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 10:18:09.749014 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 15 10:18:09.749014 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application Jun 15 10:18:09.749618 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40334b59-b45b-4d0e-8aa4-f1e71dd17501 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:18:09.749618 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40334b59-b45b-4d0e-8aa4-f1e71dd17501 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:09.749618 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40334b59-b45b-4d0e-8aa4-f1e71dd17501 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:09.749618 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40334b59-b45b-4d0e-8aa4-f1e71dd17501 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:09.751807 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 131/261] 10.222.0.49 () {56 vars in 1114 bytes} [Wed Jun 15 10:18:09 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 320 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:18:09.757612 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e757e1ba-b97d-4733-8b0b-d2d079627861 None admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:18:09.799339 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:18:09.867112 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40334b59-b45b-4d0e-8aa4-f1e71dd17501 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:18:09.869773 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40334b59-b45b-4d0e-8aa4-f1e71dd17501 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:18:09.985030 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 131/262] 10.222.0.49 () {58 vars in 1282 bytes} [Wed Jun 15 10:18:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 478 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 15 10:18:09.990566 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40334b59-b45b-4d0e-8aa4-f1e71dd17501 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:18:10.034961 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e757e1ba-b97d-4733-8b0b-d2d079627861 None admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:18:10.039335 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=100 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:18:10.086727 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:18:10.100350 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2257557-12e5-4140-b15c-f20f171f2234 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:18:10.101677 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2257557-12e5-4140-b15c-f20f171f2234 None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:10.102167 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2257557-12e5-4140-b15c-f20f171f2234 None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:10.102689 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2257557-12e5-4140-b15c-f20f171f2234 None admin] PATH_INFO: `/v3/roles` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:10.103675 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2257557-12e5-4140-b15c-f20f171f2234 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:18:10.104190 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2257557-12e5-4140-b15c-f20f171f2234 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:18:10.105809 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2257557-12e5-4140-b15c-f20f171f2234 None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:18:10.114906 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 132/263] 10.222.0.49 () {56 vars in 1111 bytes} [Wed Jun 15 10:18:09 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 358 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 10:18:10.120186 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2257557-12e5-4140-b15c-f20f171f2234 None admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:18:10.139114 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40334b59-b45b-4d0e-8aa4-f1e71dd17501 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:18:10.174708 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9419e51-ed56-46f6-b41a-5bc9c9af9e7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:18:10.174708 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9419e51-ed56-46f6-b41a-5bc9c9af9e7b None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:10.174708 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9419e51-ed56-46f6-b41a-5bc9c9af9e7b None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:10.174708 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9419e51-ed56-46f6-b41a-5bc9c9af9e7b None admin] PATH_INFO: `/v3/projects/df1f045b58c24b4dba0964eff7d2e540` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:10.175574 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9419e51-ed56-46f6-b41a-5bc9c9af9e7b None admin] RBAC: Authorizing `identity:get_project(project_id=df1f045b58c24b4dba0964eff7d2e540)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:18:10.175574 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9419e51-ed56-46f6-b41a-5bc9c9af9e7b None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:18:10.177892 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 132/264] 10.222.0.49 () {56 vars in 1186 bytes} [Wed Jun 15 10:18:09 2022] GET /identity/v3/projects/df1f045b58c24b4dba0964eff7d2e540 => generated 302 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:18:10.187537 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9419e51-ed56-46f6-b41a-5bc9c9af9e7b None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:18:10.304065 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9419e51-ed56-46f6-b41a-5bc9c9af9e7b None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:18:10.332609 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:18:10.342190 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b540ae-dad3-439e-832c-ca9461b51662 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:18:10.343196 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b540ae-dad3-439e-832c-ca9461b51662 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:10.343680 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b540ae-dad3-439e-832c-ca9461b51662 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:10.344165 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b540ae-dad3-439e-832c-ca9461b51662 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:10.346111 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b540ae-dad3-439e-832c-ca9461b51662 None admin] RBAC: Adding query filter params (role.id=8051174e7a7947d5b991ebd793c2e543, scope.project.id=df1f045b58c24b4dba0964eff7d2e540, user.id=c3f14bfbae9a41858def110ed6fe1d8e) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:18:10.346686 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b540ae-dad3-439e-832c-ca9461b51662 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:18:10.348392 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b540ae-dad3-439e-832c-ca9461b51662 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:18:10.353238 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2257557-12e5-4140-b15c-f20f171f2234 None admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:18:10.355488 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 133/265] 10.222.0.49 () {56 vars in 1374 bytes} [Wed Jun 15 10:18:10 2022] GET /identity/v3/role_assignments?user.id=c3f14bfbae9a41858def110ed6fe1d8e&scope.project.id=df1f045b58c24b4dba0964eff7d2e540&role.id=8051174e7a7947d5b991ebd793c2e543 => generated 608 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:18:10.413257 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0e04c7-db80-4327-a6aa-1f8639bdc734 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:18:10.414158 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0e04c7-db80-4327-a6aa-1f8639bdc734 None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:10.414444 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0e04c7-db80-4327-a6aa-1f8639bdc734 None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:10.414719 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0e04c7-db80-4327-a6aa-1f8639bdc734 None admin] PATH_INFO: `/v3/users/c3f14bfbae9a41858def110ed6fe1d8e` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:10.420299 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca0e04c7-db80-4327-a6aa-1f8639bdc734 None admin] RBAC: Authorizing `identity:get_user(user_id=c3f14bfbae9a41858def110ed6fe1d8e)` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:18:10.421915 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca0e04c7-db80-4327-a6aa-1f8639bdc734 None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:18:10.424042 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 133/266] 10.222.0.49 () {56 vars in 1177 bytes} [Wed Jun 15 10:18:10 2022] GET /identity/v3/users/c3f14bfbae9a41858def110ed6fe1d8e => generated 283 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:18:10.429587 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b540ae-dad3-439e-832c-ca9461b51662 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:18:10.578541 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b540ae-dad3-439e-832c-ca9461b51662 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:18:10.613652 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97212cf-9fa7-496a-9fc5-e12238d58942 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:18:10.614652 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97212cf-9fa7-496a-9fc5-e12238d58942 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:10.615157 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97212cf-9fa7-496a-9fc5-e12238d58942 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:10.615625 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97212cf-9fa7-496a-9fc5-e12238d58942 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:10.677694 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:18:10.696188 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e97212cf-9fa7-496a-9fc5-e12238d58942 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:18:10.698119 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e97212cf-9fa7-496a-9fc5-e12238d58942 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:18:10.787799 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0e7849-d77f-46b8-9f4a-0dd9115f98ca None None] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:10.788107 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0e7849-d77f-46b8-9f4a-0dd9115f98ca None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:10.788379 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0e7849-d77f-46b8-9f4a-0dd9115f98ca None None] PATH_INFO: `/` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:10.789923 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 134/267] 10.222.0.49 () {54 vars in 926 bytes} [Wed Jun 15 10:18:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:18:10.797495 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb709963-5af7-457c-9b2e-11d4e57dfecb None None] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:10.797788 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb709963-5af7-457c-9b2e-11d4e57dfecb None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:10.798073 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb709963-5af7-457c-9b2e-11d4e57dfecb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:10.820293 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.auth.core [None req-fb709963-5af7-457c-9b2e-11d4e57dfecb None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:18:10.821932 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-fb709963-5af7-457c-9b2e-11d4e57dfecb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:18:10.860728 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 134/268] 10.222.0.49 () {58 vars in 1282 bytes} [Wed Jun 15 10:18:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 431 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 15 10:18:10.861917 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=100 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:18:10.882359 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97212cf-9fa7-496a-9fc5-e12238d58942 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:18:10.904944 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 135/269] 10.222.0.49 () {58 vars in 1023 bytes} [Wed Jun 15 10:18:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 15 10:18:10.931071 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb709963-5af7-457c-9b2e-11d4e57dfecb None None] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:18:11.041804 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb709963-5af7-457c-9b2e-11d4e57dfecb None None] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:18:11.081869 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5075d06c-2c3a-44c2-b825-80a20ae7b026 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:18:11.082999 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5075d06c-2c3a-44c2-b825-80a20ae7b026 None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:11.083515 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5075d06c-2c3a-44c2-b825-80a20ae7b026 None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:11.084010 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5075d06c-2c3a-44c2-b825-80a20ae7b026 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:11.096224 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5075d06c-2c3a-44c2-b825-80a20ae7b026 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 10:18:11.104891 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application [None req-5075d06c-2c3a-44c2-b825-80a20ae7b026 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 15 10:18:11.104891 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:18:11.104891 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:18:11.104891 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:18:11.104891 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:18:11.104891 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:18:11.104891 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:18:11.104891 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:18:11.104891 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:18:11.104891 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:18:11.104891 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:18:11.104891 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:18:11.104891 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 10:18:11.104891 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 10:18:11.104891 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 10:18:11.104891 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 10:18:11.106178 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 10:18:11.106178 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 10:18:11.106178 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 10:18:11.106178 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 10:18:11.106178 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:18:11.106178 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:18:11.106178 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:18:11.106178 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:18:11.106178 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:18:11.106178 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:18:11.106178 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:18:11.106178 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application with Lock( Jun 15 10:18:11.106178 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:18:11.106178 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:18:11.106178 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:18:11.106178 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:18:11.107035 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:18:11.107035 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:18:11.107035 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:18:11.107035 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:18:11.107035 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 10:18:11.107035 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 10:18:11.107035 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 10:18:11.107035 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 10:18:11.107035 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 10:18:11.107035 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 10:18:11.107035 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 15 10:18:11.107035 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application Jun 15 10:18:11.109791 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 136/270] 10.222.0.49 () {56 vars in 1099 bytes} [Wed Jun 15 10:18:10 2022] GET /identity/v3/roles/member => generated 84 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 10:18:11.116105 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5075d06c-2c3a-44c2-b825-80a20ae7b026 None admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:18:11.145114 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:18:11.145313 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=100 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:18:11.178941 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97212cf-9fa7-496a-9fc5-e12238d58942 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:18:11.229712 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5075d06c-2c3a-44c2-b825-80a20ae7b026 None admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:18:11.270196 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a83872-0d32-421b-8270-d5e8f2207861 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:18:11.271295 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a83872-0d32-421b-8270-d5e8f2207861 None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:11.272068 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a83872-0d32-421b-8270-d5e8f2207861 None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:11.272567 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a83872-0d32-421b-8270-d5e8f2207861 None admin] PATH_INFO: `/v3/roles` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:11.273507 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a83872-0d32-421b-8270-d5e8f2207861 None admin] RBAC: Adding query filter params (name=member) {{(pid=80119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:18:11.274013 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a83872-0d32-421b-8270-d5e8f2207861 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:18:11.275659 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a83872-0d32-421b-8270-d5e8f2207861 None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:18:11.282657 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c4a9b8-1bd2-4176-bf46-451485deb5fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:18:11.283757 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c4a9b8-1bd2-4176-bf46-451485deb5fb None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:11.284180 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c4a9b8-1bd2-4176-bf46-451485deb5fb None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:11.284591 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c4a9b8-1bd2-4176-bf46-451485deb5fb None admin] PATH_INFO: `/v3/projects/df1f045b58c24b4dba0964eff7d2e540` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:11.285506 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 137/271] 10.222.0.49 () {56 vars in 1101 bytes} [Wed Jun 15 10:18:11 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:18:11.287022 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a8b505-86eb-4e89-b04d-f999ebc279b9 None None] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:11.287556 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a8b505-86eb-4e89-b04d-f999ebc279b9 None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:11.288033 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a8b505-86eb-4e89-b04d-f999ebc279b9 None None] PATH_INFO: `/` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:11.289474 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 138/272] 10.222.0.49 () {54 vars in 926 bytes} [Wed Jun 15 10:18:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:18:11.291601 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c4a9b8-1bd2-4176-bf46-451485deb5fb None admin] RBAC: Authorizing `identity:get_project(project_id=df1f045b58c24b4dba0964eff7d2e540)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:18:11.292705 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a8b505-86eb-4e89-b04d-f999ebc279b9 None None] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:18:11.292812 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c4a9b8-1bd2-4176-bf46-451485deb5fb None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:18:11.297562 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 135/273] 10.222.0.49 () {56 vars in 1186 bytes} [Wed Jun 15 10:18:10 2022] GET /identity/v3/projects/df1f045b58c24b4dba0964eff7d2e540 => generated 302 bytes in 415 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:18:11.298189 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eed0673-17b1-4456-8bd8-60a5e4edab8d None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:11.303167 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eed0673-17b1-4456-8bd8-60a5e4edab8d None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:11.303167 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eed0673-17b1-4456-8bd8-60a5e4edab8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:11.323529 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-3eed0673-17b1-4456-8bd8-60a5e4edab8d None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:18:11.325108 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-3eed0673-17b1-4456-8bd8-60a5e4edab8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:18:11.391491 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=100 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:18:11.403242 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 136/274] 10.222.0.49 () {58 vars in 1023 bytes} [Wed Jun 15 10:18:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 10:18:11.405281 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eed0673-17b1-4456-8bd8-60a5e4edab8d None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:18:11.417845 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a8b505-86eb-4e89-b04d-f999ebc279b9 None None] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:18:11.474768 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0677130-04d3-4589-bc3b-1a5ccc268523 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:18:11.475691 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0677130-04d3-4589-bc3b-1a5ccc268523 None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:11.475974 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0677130-04d3-4589-bc3b-1a5ccc268523 None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:11.476248 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0677130-04d3-4589-bc3b-1a5ccc268523 None admin] PATH_INFO: `/v3/users/9529db0ddca44678bd92a22b908cba6f` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:11.491338 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0677130-04d3-4589-bc3b-1a5ccc268523 None admin] RBAC: Authorizing `identity:get_user(user_id=9529db0ddca44678bd92a22b908cba6f)` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:18:11.492823 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0677130-04d3-4589-bc3b-1a5ccc268523 None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:18:11.499879 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 139/275] 10.222.0.49 () {56 vars in 1177 bytes} [Wed Jun 15 10:18:11 2022] GET /identity/v3/users/9529db0ddca44678bd92a22b908cba6f => generated 293 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:18:11.500733 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0677130-04d3-4589-bc3b-1a5ccc268523 None admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:18:11.513708 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:18:11.526369 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eed0673-17b1-4456-8bd8-60a5e4edab8d None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:18:11.563268 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00f8d24-91fa-4350-9f70-7e479b1a47a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:18:11.563939 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00f8d24-91fa-4350-9f70-7e479b1a47a1 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:11.564217 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00f8d24-91fa-4350-9f70-7e479b1a47a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:11.564494 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00f8d24-91fa-4350-9f70-7e479b1a47a1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:11.565309 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a00f8d24-91fa-4350-9f70-7e479b1a47a1 None admin] RBAC: Adding query filter params (role.id=4d2118569d074a288b37d120110c300d, scope.project.id=df1f045b58c24b4dba0964eff7d2e540, user.id=c3f14bfbae9a41858def110ed6fe1d8e) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:18:11.565615 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a00f8d24-91fa-4350-9f70-7e479b1a47a1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:18:11.567057 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a00f8d24-91fa-4350-9f70-7e479b1a47a1 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:18:11.578958 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 137/276] 10.222.0.49 () {56 vars in 1374 bytes} [Wed Jun 15 10:18:11 2022] GET /identity/v3/role_assignments?user.id=c3f14bfbae9a41858def110ed6fe1d8e&scope.project.id=df1f045b58c24b4dba0964eff7d2e540&role.id=4d2118569d074a288b37d120110c300d => generated 608 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:18:11.578958 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00f8d24-91fa-4350-9f70-7e479b1a47a1 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:18:11.672401 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0677130-04d3-4589-bc3b-1a5ccc268523 None admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:18:11.738394 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00f8d24-91fa-4350-9f70-7e479b1a47a1 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:18:11.741870 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4a1d5c-5ac1-45f2-b054-d19052103d8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:18:11.742751 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4a1d5c-5ac1-45f2-b054-d19052103d8f None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:11.743043 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4a1d5c-5ac1-45f2-b054-d19052103d8f None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:11.743353 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4a1d5c-5ac1-45f2-b054-d19052103d8f None admin] PATH_INFO: `/v3/domains/default` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:11.746211 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc4a1d5c-5ac1-45f2-b054-d19052103d8f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:18:11.748448 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc4a1d5c-5ac1-45f2-b054-d19052103d8f None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:18:11.750277 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 140/277] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Jun 15 10:18:11 2022] GET /identity/v3/domains/default => generated 199 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:18:11.756062 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4a1d5c-5ac1-45f2-b054-d19052103d8f None admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:18:11.779840 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=100 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:18:11.783310 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4dabede-02d6-4b42-ae4b-a01372ce999e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:18:11.783983 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4dabede-02d6-4b42-ae4b-a01372ce999e None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:11.784266 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4dabede-02d6-4b42-ae4b-a01372ce999e None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:11.784536 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4dabede-02d6-4b42-ae4b-a01372ce999e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:11.785376 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4dabede-02d6-4b42-ae4b-a01372ce999e None admin] RBAC: Adding query filter params (role.id=6876e57cc51c448097ab109d35b2a3da, user.id=9529db0ddca44678bd92a22b908cba6f, scope.system=all) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:18:11.785685 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4dabede-02d6-4b42-ae4b-a01372ce999e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:18:11.787410 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4dabede-02d6-4b42-ae4b-a01372ce999e None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:18:11.799327 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 138/278] 10.222.0.49 () {56 vars in 1308 bytes} [Wed Jun 15 10:18:11 2022] GET /identity/v3/role_assignments?user.id=9529db0ddca44678bd92a22b908cba6f&scope.system=all&role.id=6876e57cc51c448097ab109d35b2a3da => generated 226 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:18:11.888264 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4a1d5c-5ac1-45f2-b054-d19052103d8f None admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:18:11.933148 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c42ab05-ed45-474b-970d-7509e2908d76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:18:11.933791 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c42ab05-ed45-474b-970d-7509e2908d76 None admin] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:11.934082 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c42ab05-ed45-474b-970d-7509e2908d76 None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:11.934363 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c42ab05-ed45-474b-970d-7509e2908d76 None admin] PATH_INFO: `/v3/users` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:11.935194 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c42ab05-ed45-474b-970d-7509e2908d76 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:18:11.936865 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c42ab05-ed45-474b-970d-7509e2908d76 None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:18:11.964803 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 141/279] 10.222.0.49 () {60 vars in 1132 bytes} [Wed Jun 15 10:18:11 2022] POST /identity/v3/users => generated 293 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 10:18:12.014484 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0643f4-6477-445c-862a-89eaac2af019 None None] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:12.014832 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0643f4-6477-445c-862a-89eaac2af019 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:12.015033 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0643f4-6477-445c-862a-89eaac2af019 None None] PATH_INFO: `/` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:12.016640 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 139/280] 10.222.0.49 () {54 vars in 926 bytes} [Wed Jun 15 10:18:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:18:12.022479 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acba9731-6fcc-43cb-810a-ab84433a33a0 None None] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:12.022774 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acba9731-6fcc-43cb-810a-ab84433a33a0 None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:12.023054 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acba9731-6fcc-43cb-810a-ab84433a33a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:12.046887 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.auth.core [None req-acba9731-6fcc-43cb-810a-ab84433a33a0 None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:18:12.048549 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-acba9731-6fcc-43cb-810a-ab84433a33a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:18:12.132949 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:18:12.144753 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 142/281] 10.222.0.49 () {58 vars in 1023 bytes} [Wed Jun 15 10:18:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 10:18:12.168352 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0643f4-6477-445c-862a-89eaac2af019 None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:18:12.190390 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:18:12.282410 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0643f4-6477-445c-862a-89eaac2af019 None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:18:12.323609 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5bb5487-400c-4ba0-969a-ddb955d975b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:18:12.324440 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bb5487-400c-4ba0-969a-ddb955d975b5 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:12.324728 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bb5487-400c-4ba0-969a-ddb955d975b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:12.325005 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bb5487-400c-4ba0-969a-ddb955d975b5 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:12.339362 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5bb5487-400c-4ba0-969a-ddb955d975b5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 10:18:12.348007 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application [None req-a5bb5487-400c-4ba0-969a-ddb955d975b5 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 15 10:18:12.348007 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:18:12.348007 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:18:12.348007 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:18:12.348007 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:18:12.348007 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:18:12.348007 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:18:12.348007 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:18:12.348007 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:18:12.348007 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:18:12.348007 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:18:12.348007 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:18:12.348007 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 10:18:12.348007 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 10:18:12.348007 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 10:18:12.348007 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 10:18:12.348910 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 10:18:12.348910 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 10:18:12.348910 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 10:18:12.348910 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 10:18:12.348910 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:18:12.348910 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:18:12.348910 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:18:12.348910 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:18:12.348910 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:18:12.348910 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:18:12.348910 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:18:12.348910 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application with Lock( Jun 15 10:18:12.348910 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:18:12.348910 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:18:12.348910 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:18:12.348910 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:18:12.349706 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:18:12.349706 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:18:12.349706 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:18:12.349706 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:18:12.349706 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 10:18:12.349706 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 10:18:12.349706 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 10:18:12.349706 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 10:18:12.349706 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 10:18:12.349706 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 10:18:12.349706 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 15 10:18:12.349706 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Jun 15 10:18:12.349706 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 140/282] 10.222.0.49 () {56 vars in 1099 bytes} [Wed Jun 15 10:18:12 2022] GET /identity/v3/roles/reader => generated 84 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:18:12.355861 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acba9731-6fcc-43cb-810a-ab84433a33a0 None None] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:18:12.467882 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acba9731-6fcc-43cb-810a-ab84433a33a0 None None] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:18:12.487020 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2fd51b-e90c-494b-b04a-53de5ce38e7d None None] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:12.487561 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2fd51b-e90c-494b-b04a-53de5ce38e7d None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:12.488069 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2fd51b-e90c-494b-b04a-53de5ce38e7d None None] PATH_INFO: `/` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:12.492183 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 141/283] 10.222.0.49 () {54 vars in 926 bytes} [Wed Jun 15 10:18:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 15 10:18:12.494617 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637a957a-ab1f-40d2-b720-82d0370543e9 None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:12.495141 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637a957a-ab1f-40d2-b720-82d0370543e9 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:12.495615 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637a957a-ab1f-40d2-b720-82d0370543e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:12.506170 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc914777-9e98-46e2-bde9-e0e4df8ba3bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:18:12.506850 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc914777-9e98-46e2-bde9-e0e4df8ba3bd None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:12.507237 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc914777-9e98-46e2-bde9-e0e4df8ba3bd None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:12.507522 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc914777-9e98-46e2-bde9-e0e4df8ba3bd None admin] PATH_INFO: `/v3/roles` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:12.508283 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc914777-9e98-46e2-bde9-e0e4df8ba3bd None admin] RBAC: Adding query filter params (name=reader) {{(pid=80119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:18:12.508591 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc914777-9e98-46e2-bde9-e0e4df8ba3bd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:18:12.510063 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc914777-9e98-46e2-bde9-e0e4df8ba3bd None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:18:12.521472 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 143/284] 10.222.0.49 () {56 vars in 1101 bytes} [Wed Jun 15 10:18:12 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:18:12.530457 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc914777-9e98-46e2-bde9-e0e4df8ba3bd None admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:18:12.530672 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-637a957a-ab1f-40d2-b720-82d0370543e9 None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:18:12.531921 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-637a957a-ab1f-40d2-b720-82d0370543e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:18:12.652750 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=100 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:18:12.666463 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 142/285] 10.222.0.49 () {58 vars in 1023 bytes} [Wed Jun 15 10:18:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 15 10:18:12.700111 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637a957a-ab1f-40d2-b720-82d0370543e9 None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:18:12.711439 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:18:12.716056 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc914777-9e98-46e2-bde9-e0e4df8ba3bd None admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:18:12.820385 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8716c6f-254d-426b-89bb-978b253fbc97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:18:12.821294 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8716c6f-254d-426b-89bb-978b253fbc97 None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:12.821588 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8716c6f-254d-426b-89bb-978b253fbc97 None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:12.821864 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8716c6f-254d-426b-89bb-978b253fbc97 None admin] PATH_INFO: `/v3/users/b142a76f3f994d00b76f1eecb82a7541` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:12.823106 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8716c6f-254d-426b-89bb-978b253fbc97 None admin] RBAC: Authorizing `identity:get_user(user_id=b142a76f3f994d00b76f1eecb82a7541)` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:18:12.824595 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8716c6f-254d-426b-89bb-978b253fbc97 None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:18:12.827226 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 144/286] 10.222.0.49 () {56 vars in 1177 bytes} [Wed Jun 15 10:18:12 2022] GET /identity/v3/users/b142a76f3f994d00b76f1eecb82a7541 => generated 297 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 10:18:12.835987 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8716c6f-254d-426b-89bb-978b253fbc97 None admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:18:12.838114 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637a957a-ab1f-40d2-b720-82d0370543e9 None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:18:12.876293 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5cc259d-e862-4d3c-b5b4-53e2cda53ee9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:18:12.877104 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cc259d-e862-4d3c-b5b4-53e2cda53ee9 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:12.877390 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cc259d-e862-4d3c-b5b4-53e2cda53ee9 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:12.877663 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cc259d-e862-4d3c-b5b4-53e2cda53ee9 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:12.889827 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5cc259d-e862-4d3c-b5b4-53e2cda53ee9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 10:18:12.905055 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application [None req-d5cc259d-e862-4d3c-b5b4-53e2cda53ee9 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 15 10:18:12.905055 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:18:12.905055 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:18:12.905055 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:18:12.905055 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:18:12.905055 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:18:12.905055 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:18:12.905055 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:18:12.905055 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:18:12.905055 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:18:12.905055 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:18:12.905055 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:18:12.905055 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 10:18:12.905055 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 10:18:12.905055 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 10:18:12.905055 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 10:18:12.905954 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 10:18:12.905954 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 10:18:12.905954 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 10:18:12.905954 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 10:18:12.905954 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:18:12.905954 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:18:12.905954 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:18:12.905954 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:18:12.905954 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:18:12.905954 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:18:12.905954 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:18:12.905954 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application with Lock( Jun 15 10:18:12.905954 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:18:12.905954 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:18:12.905954 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:18:12.905954 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:18:12.906977 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:18:12.906977 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:18:12.906977 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:18:12.906977 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:18:12.906977 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 10:18:12.906977 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 10:18:12.906977 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 10:18:12.906977 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 10:18:12.906977 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 10:18:12.906977 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 10:18:12.906977 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 15 10:18:12.906977 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Jun 15 10:18:12.906977 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 143/287] 10.222.0.49 () {56 vars in 1099 bytes} [Wed Jun 15 10:18:12 2022] GET /identity/v3/roles/member => generated 84 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 10:18:12.912421 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5cc259d-e862-4d3c-b5b4-53e2cda53ee9 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:18:13.035460 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:18:13.059513 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8716c6f-254d-426b-89bb-978b253fbc97 None admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:18:13.097345 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5cc259d-e862-4d3c-b5b4-53e2cda53ee9 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:18:13.107342 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=100 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:18:13.122121 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8417ce-5907-474f-8883-a9061f17cd8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:18:13.123605 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8417ce-5907-474f-8883-a9061f17cd8d None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:13.123894 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8417ce-5907-474f-8883-a9061f17cd8d None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:13.124173 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8417ce-5907-474f-8883-a9061f17cd8d None admin] PATH_INFO: `/v3/projects/df1f045b58c24b4dba0964eff7d2e540` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:13.126616 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb8417ce-5907-474f-8883-a9061f17cd8d None admin] RBAC: Authorizing `identity:get_project(project_id=df1f045b58c24b4dba0964eff7d2e540)` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:18:13.128112 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb8417ce-5907-474f-8883-a9061f17cd8d None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:18:13.139156 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 145/288] 10.222.0.49 () {56 vars in 1186 bytes} [Wed Jun 15 10:18:12 2022] GET /identity/v3/projects/df1f045b58c24b4dba0964eff7d2e540 => generated 302 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:18:13.140728 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8417ce-5907-474f-8883-a9061f17cd8d None admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:18:13.159168 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8be7b80-cdb9-4bbb-8c98-309b5d6a52cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:18:13.160159 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8be7b80-cdb9-4bbb-8c98-309b5d6a52cd None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:13.160645 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8be7b80-cdb9-4bbb-8c98-309b5d6a52cd None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:13.161117 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8be7b80-cdb9-4bbb-8c98-309b5d6a52cd None admin] PATH_INFO: `/v3/roles` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:13.162078 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8be7b80-cdb9-4bbb-8c98-309b5d6a52cd None admin] RBAC: Adding query filter params (name=member) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:18:13.162584 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8be7b80-cdb9-4bbb-8c98-309b5d6a52cd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:18:13.165396 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8be7b80-cdb9-4bbb-8c98-309b5d6a52cd None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:18:13.180595 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 144/289] 10.222.0.49 () {56 vars in 1101 bytes} [Wed Jun 15 10:18:12 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:18:13.184415 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8be7b80-cdb9-4bbb-8c98-309b5d6a52cd None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:18:13.323930 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8417ce-5907-474f-8883-a9061f17cd8d None admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:18:13.359141 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8be7b80-cdb9-4bbb-8c98-309b5d6a52cd None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:18:13.384906 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb09cf2-ffca-4ea7-a959-9eb36ea8c83a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:18:13.386226 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb09cf2-ffca-4ea7-a959-9eb36ea8c83a None admin] REQUEST_METHOD: `PUT` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:13.387278 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb09cf2-ffca-4ea7-a959-9eb36ea8c83a None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:13.388542 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb09cf2-ffca-4ea7-a959-9eb36ea8c83a None admin] PATH_INFO: `/v3/projects/df1f045b58c24b4dba0964eff7d2e540/users/b142a76f3f994d00b76f1eecb82a7541/roles/301a3c8a23ba40edbc35df1da8af0de9` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:13.392096 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb09cf2-ffca-4ea7-a959-9eb36ea8c83a None admin] RBAC: Authorizing `identity:create_grant(project_id=df1f045b58c24b4dba0964eff7d2e540, user_id=b142a76f3f994d00b76f1eecb82a7541, role_id=301a3c8a23ba40edbc35df1da8af0de9)` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:18:13.395403 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb09cf2-ffca-4ea7-a959-9eb36ea8c83a None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:18:13.405080 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:18:13.416614 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 146/290] 10.222.0.49 () {58 vars in 1439 bytes} [Wed Jun 15 10:18:13 2022] PUT /identity/v3/projects/df1f045b58c24b4dba0964eff7d2e540/users/b142a76f3f994d00b76f1eecb82a7541/roles/301a3c8a23ba40edbc35df1da8af0de9 => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:18:13.429282 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebb7e78-4f1d-4e74-9098-2e10894a2460 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:18:13.430207 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebb7e78-4f1d-4e74-9098-2e10894a2460 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:13.430496 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebb7e78-4f1d-4e74-9098-2e10894a2460 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:13.430790 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebb7e78-4f1d-4e74-9098-2e10894a2460 None admin] PATH_INFO: `/v3/users/4bef473873954d20ad88a891d1c0f60e` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:13.432172 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bebb7e78-4f1d-4e74-9098-2e10894a2460 None admin] RBAC: Authorizing `identity:get_user(user_id=4bef473873954d20ad88a891d1c0f60e)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:18:13.433672 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bebb7e78-4f1d-4e74-9098-2e10894a2460 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:18:13.444100 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 145/291] 10.222.0.49 () {56 vars in 1177 bytes} [Wed Jun 15 10:18:13 2022] GET /identity/v3/users/4bef473873954d20ad88a891d1c0f60e => generated 297 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:18:13.449664 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb09cf2-ffca-4ea7-a959-9eb36ea8c83a None admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:18:13.551705 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:18:13.580406 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb09cf2-ffca-4ea7-a959-9eb36ea8c83a None admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:18:13.621562 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9d64f0-c9fc-4930-afac-af984fb27ef9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:18:13.622653 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9d64f0-c9fc-4930-afac-af984fb27ef9 None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:13.623176 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9d64f0-c9fc-4930-afac-af984fb27ef9 None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:13.623703 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9d64f0-c9fc-4930-afac-af984fb27ef9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:13.698476 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc9d64f0-c9fc-4930-afac-af984fb27ef9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:18:13.701013 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc9d64f0-c9fc-4930-afac-af984fb27ef9 None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:18:13.757459 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=100 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:18:13.806114 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 147/292] 10.222.0.49 () {58 vars in 1282 bytes} [Wed Jun 15 10:18:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 357 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 15 10:18:13.811337 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebb7e78-4f1d-4e74-9098-2e10894a2460 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:18:13.914695 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebb7e78-4f1d-4e74-9098-2e10894a2460 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:18:13.948733 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b589786a-e9f3-4c36-9d11-1ad77d4f2747 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:18:13.949616 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b589786a-e9f3-4c36-9d11-1ad77d4f2747 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:13.949962 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b589786a-e9f3-4c36-9d11-1ad77d4f2747 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:13.950243 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b589786a-e9f3-4c36-9d11-1ad77d4f2747 None admin] PATH_INFO: `/v3/projects/df1f045b58c24b4dba0964eff7d2e540` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:13.952053 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b589786a-e9f3-4c36-9d11-1ad77d4f2747 None admin] RBAC: Authorizing `identity:get_project(project_id=df1f045b58c24b4dba0964eff7d2e540)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:18:13.953523 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b589786a-e9f3-4c36-9d11-1ad77d4f2747 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:18:13.955563 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 146/293] 10.222.0.49 () {56 vars in 1186 bytes} [Wed Jun 15 10:18:13 2022] GET /identity/v3/projects/df1f045b58c24b4dba0964eff7d2e540 => generated 302 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 10:18:13.964206 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9d64f0-c9fc-4930-afac-af984fb27ef9 None admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:18:14.088246 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9d64f0-c9fc-4930-afac-af984fb27ef9 None admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:18:14.140842 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b68edc-1559-4577-ac74-fdb41e963787 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:18:14.141604 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b68edc-1559-4577-ac74-fdb41e963787 None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:14.141867 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b68edc-1559-4577-ac74-fdb41e963787 None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:14.142153 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b68edc-1559-4577-ac74-fdb41e963787 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:14.143062 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b68edc-1559-4577-ac74-fdb41e963787 None admin] RBAC: Adding query filter params (role.id=6876e57cc51c448097ab109d35b2a3da, scope.project.id=df1f045b58c24b4dba0964eff7d2e540, user.id=4bef473873954d20ad88a891d1c0f60e) {{(pid=80119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:18:14.143410 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b68edc-1559-4577-ac74-fdb41e963787 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:18:14.144950 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b68edc-1559-4577-ac74-fdb41e963787 None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:18:14.151906 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 148/294] 10.222.0.49 () {56 vars in 1374 bytes} [Wed Jun 15 10:18:13 2022] GET /identity/v3/role_assignments?user.id=4bef473873954d20ad88a891d1c0f60e&scope.project.id=df1f045b58c24b4dba0964eff7d2e540&role.id=6876e57cc51c448097ab109d35b2a3da => generated 608 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:18:14.191280 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39050cbb-5113-480f-9216-6a70c74f869b None None] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:14.191568 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39050cbb-5113-480f-9216-6a70c74f869b None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:14.191774 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39050cbb-5113-480f-9216-6a70c74f869b None None] PATH_INFO: `/` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:14.193167 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 147/295] 10.222.0.49 () {54 vars in 926 bytes} [Wed Jun 15 10:18:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:18:14.198793 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754268d3-498e-493d-ae8d-be2c02c87912 None None] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:14.199139 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754268d3-498e-493d-ae8d-be2c02c87912 None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:14.199402 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754268d3-498e-493d-ae8d-be2c02c87912 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:14.226905 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.auth.core [None req-754268d3-498e-493d-ae8d-be2c02c87912 None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:18:14.228194 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-754268d3-498e-493d-ae8d-be2c02c87912 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:18:14.251355 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=100 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:18:14.329898 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 149/296] 10.222.0.49 () {58 vars in 1023 bytes} [Wed Jun 15 10:18:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 10:18:14.362465 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39050cbb-5113-480f-9216-6a70c74f869b None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:18:14.440053 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:18:14.482939 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39050cbb-5113-480f-9216-6a70c74f869b None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:18:14.519135 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0549f19c-98dc-4013-8ac1-762ecdbcd8b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:18:14.519954 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0549f19c-98dc-4013-8ac1-762ecdbcd8b9 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:14.520239 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0549f19c-98dc-4013-8ac1-762ecdbcd8b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:14.520517 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0549f19c-98dc-4013-8ac1-762ecdbcd8b9 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:14.532164 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0549f19c-98dc-4013-8ac1-762ecdbcd8b9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 10:18:14.540088 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application [None req-0549f19c-98dc-4013-8ac1-762ecdbcd8b9 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 15 10:18:14.540088 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:18:14.540088 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:18:14.540088 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:18:14.540088 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:18:14.540088 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:18:14.540088 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:18:14.540088 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:18:14.540088 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:18:14.540088 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:18:14.540088 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:18:14.540088 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:18:14.540088 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 10:18:14.540088 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 10:18:14.540088 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 10:18:14.540088 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 10:18:14.540995 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 10:18:14.540995 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 10:18:14.540995 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 10:18:14.540995 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 10:18:14.540995 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:18:14.540995 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:18:14.540995 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:18:14.540995 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:18:14.540995 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:18:14.540995 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:18:14.540995 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:18:14.540995 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application with Lock( Jun 15 10:18:14.540995 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:18:14.540995 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:18:14.540995 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:18:14.540995 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:18:14.541809 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:18:14.541809 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:18:14.541809 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:18:14.541809 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:18:14.541809 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 10:18:14.541809 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 10:18:14.541809 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 10:18:14.541809 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 10:18:14.541809 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 10:18:14.541809 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 10:18:14.541809 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 15 10:18:14.541809 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Jun 15 10:18:14.541809 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 148/297] 10.222.0.49 () {56 vars in 1099 bytes} [Wed Jun 15 10:18:14 2022] GET /identity/v3/roles/member => generated 84 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:18:14.548254 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754268d3-498e-493d-ae8d-be2c02c87912 None None] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:18:14.675356 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754268d3-498e-493d-ae8d-be2c02c87912 None None] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:18:14.689368 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89badf18-0ba9-4c41-ae08-5fe9760f4c8d None None] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:14.689663 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89badf18-0ba9-4c41-ae08-5fe9760f4c8d None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:14.689947 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89badf18-0ba9-4c41-ae08-5fe9760f4c8d None None] PATH_INFO: `/` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:14.691628 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 149/298] 10.222.0.49 () {54 vars in 926 bytes} [Wed Jun 15 10:18:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:18:14.696819 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149511ba-17b4-4887-9d6a-2e13000f4957 None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:14.697120 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149511ba-17b4-4887-9d6a-2e13000f4957 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:14.697386 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149511ba-17b4-4887-9d6a-2e13000f4957 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:14.715513 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-149511ba-17b4-4887-9d6a-2e13000f4957 None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:18:14.716764 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-149511ba-17b4-4887-9d6a-2e13000f4957 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:18:14.717193 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ffa341-648f-4a47-a040-1d697417e4dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:18:14.717892 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ffa341-648f-4a47-a040-1d697417e4dc None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:14.718182 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ffa341-648f-4a47-a040-1d697417e4dc None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:14.718462 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ffa341-648f-4a47-a040-1d697417e4dc None admin] PATH_INFO: `/v3/roles` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:14.719261 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ffa341-648f-4a47-a040-1d697417e4dc None admin] RBAC: Adding query filter params (name=member) {{(pid=80119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:18:14.719574 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ffa341-648f-4a47-a040-1d697417e4dc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:18:14.720994 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ffa341-648f-4a47-a040-1d697417e4dc None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:18:14.733842 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 150/299] 10.222.0.49 () {56 vars in 1101 bytes} [Wed Jun 15 10:18:14 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:18:14.739459 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ffa341-648f-4a47-a040-1d697417e4dc None admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:18:14.754268 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=100 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:18:14.812339 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 150/300] 10.222.0.49 () {58 vars in 1023 bytes} [Wed Jun 15 10:18:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 15 10:18:14.813736 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e90ff1d-a44a-48b4-b323-5223003fe1bf None None] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:14.814029 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e90ff1d-a44a-48b4-b323-5223003fe1bf None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:14.814305 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e90ff1d-a44a-48b4-b323-5223003fe1bf None None] PATH_INFO: `/` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:14.815722 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 151/301] 10.222.0.49 () {54 vars in 926 bytes} [Wed Jun 15 10:18:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:18:14.824454 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46531d53-b0fc-4ee3-961e-242f150eae35 None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:14.824742 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46531d53-b0fc-4ee3-961e-242f150eae35 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:14.825023 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46531d53-b0fc-4ee3-961e-242f150eae35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:14.845995 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-46531d53-b0fc-4ee3-961e-242f150eae35 None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:18:14.847889 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-46531d53-b0fc-4ee3-961e-242f150eae35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:18:14.855864 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ffa341-648f-4a47-a040-1d697417e4dc None admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:18:14.889108 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733fe414-7ef7-49ee-9881-ab42ade3ff03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:18:14.889984 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733fe414-7ef7-49ee-9881-ab42ade3ff03 None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:14.890271 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733fe414-7ef7-49ee-9881-ab42ade3ff03 None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:14.890548 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733fe414-7ef7-49ee-9881-ab42ade3ff03 None admin] PATH_INFO: `/v3/users/9529db0ddca44678bd92a22b908cba6f` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:14.891782 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733fe414-7ef7-49ee-9881-ab42ade3ff03 None admin] RBAC: Authorizing `identity:get_user(user_id=9529db0ddca44678bd92a22b908cba6f)` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:18:14.893227 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733fe414-7ef7-49ee-9881-ab42ade3ff03 None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:18:14.894860 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 151/302] 10.222.0.49 () {56 vars in 1177 bytes} [Wed Jun 15 10:18:14 2022] GET /identity/v3/users/9529db0ddca44678bd92a22b908cba6f => generated 293 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 10:18:14.895653 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733fe414-7ef7-49ee-9881-ab42ade3ff03 None admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:18:14.907843 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:18:14.926413 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 152/303] 10.222.0.49 () {58 vars in 1023 bytes} [Wed Jun 15 10:18:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 15 10:18:14.931984 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46531d53-b0fc-4ee3-961e-242f150eae35 None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:18:14.981178 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=100 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:18:15.004553 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733fe414-7ef7-49ee-9881-ab42ade3ff03 None admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:18:15.039006 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6354f6d8-a04d-46e6-9102-62fa24cf69fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:18:15.039832 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6354f6d8-a04d-46e6-9102-62fa24cf69fc None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:15.040119 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6354f6d8-a04d-46e6-9102-62fa24cf69fc None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:15.040218 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46531d53-b0fc-4ee3-961e-242f150eae35 None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:18:15.040393 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6354f6d8-a04d-46e6-9102-62fa24cf69fc None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:15.051172 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6354f6d8-a04d-46e6-9102-62fa24cf69fc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 10:18:15.059625 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application [None req-6354f6d8-a04d-46e6-9102-62fa24cf69fc None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 15 10:18:15.059625 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:18:15.059625 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:18:15.059625 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:18:15.059625 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:18:15.059625 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:18:15.059625 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:18:15.059625 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:18:15.059625 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:18:15.059625 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:18:15.059625 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:18:15.059625 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:18:15.059625 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 10:18:15.059625 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 10:18:15.059625 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 10:18:15.059625 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 10:18:15.061089 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 10:18:15.061089 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 10:18:15.061089 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 10:18:15.061089 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 10:18:15.061089 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:18:15.061089 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:18:15.061089 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:18:15.061089 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:18:15.061089 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:18:15.061089 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:18:15.061089 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:18:15.061089 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application with Lock( Jun 15 10:18:15.061089 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:18:15.061089 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:18:15.061089 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:18:15.061089 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:18:15.067856 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:18:15.067856 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:18:15.067856 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:18:15.067856 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:18:15.067856 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 10:18:15.067856 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 10:18:15.067856 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 10:18:15.067856 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 10:18:15.067856 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 10:18:15.067856 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 10:18:15.067856 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 15 10:18:15.067856 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application Jun 15 10:18:15.067856 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 152/304] 10.222.0.49 () {56 vars in 1099 bytes} [Wed Jun 15 10:18:14 2022] GET /identity/v3/roles/reader => generated 84 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:18:15.067856 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6354f6d8-a04d-46e6-9102-62fa24cf69fc None admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:18:15.084629 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de14daf-0fd2-4fae-b389-53d4790dab01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:18:15.085599 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de14daf-0fd2-4fae-b389-53d4790dab01 None admin] REQUEST_METHOD: `PUT` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:15.085945 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de14daf-0fd2-4fae-b389-53d4790dab01 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:15.086285 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de14daf-0fd2-4fae-b389-53d4790dab01 None admin] PATH_INFO: `/v3/system/users/9529db0ddca44678bd92a22b908cba6f/roles/6876e57cc51c448097ab109d35b2a3da` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:15.087831 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de14daf-0fd2-4fae-b389-53d4790dab01 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=9529db0ddca44678bd92a22b908cba6f, role_id=6876e57cc51c448097ab109d35b2a3da)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:18:15.089216 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de14daf-0fd2-4fae-b389-53d4790dab01 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:18:15.107124 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 153/305] 10.222.0.49 () {58 vars in 1334 bytes} [Wed Jun 15 10:18:14 2022] PUT /identity/v3/system/users/9529db0ddca44678bd92a22b908cba6f/roles/6876e57cc51c448097ab109d35b2a3da => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:18:15.107124 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de14daf-0fd2-4fae-b389-53d4790dab01 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:18:15.172216 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:18:15.210292 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6354f6d8-a04d-46e6-9102-62fa24cf69fc None admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:18:15.224655 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de14daf-0fd2-4fae-b389-53d4790dab01 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:18:15.257626 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646752cc-7c40-4f9f-9f37-578f45129ee4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:18:15.259237 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646752cc-7c40-4f9f-9f37-578f45129ee4 None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:15.259799 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646752cc-7c40-4f9f-9f37-578f45129ee4 None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:15.260250 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646752cc-7c40-4f9f-9f37-578f45129ee4 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:15.261738 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-646752cc-7c40-4f9f-9f37-578f45129ee4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:18:15.263576 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-646752cc-7c40-4f9f-9f37-578f45129ee4 None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:18:15.263988 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9bc2f2-34ee-4140-8bc8-8d664485b054 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:18:15.264734 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9bc2f2-34ee-4140-8bc8-8d664485b054 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:15.265085 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9bc2f2-34ee-4140-8bc8-8d664485b054 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:15.265429 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9bc2f2-34ee-4140-8bc8-8d664485b054 None admin] PATH_INFO: `/v3/roles` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:15.265783 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 153/306] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Jun 15 10:18:15 2022] GET /identity/v3/domains/default => generated 199 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:18:15.266433 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9bc2f2-34ee-4140-8bc8-8d664485b054 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:18:15.266852 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9bc2f2-34ee-4140-8bc8-8d664485b054 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:18:15.268336 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9bc2f2-34ee-4140-8bc8-8d664485b054 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:18:15.272096 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646752cc-7c40-4f9f-9f37-578f45129ee4 None admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:18:15.278965 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 154/307] 10.222.0.49 () {56 vars in 1101 bytes} [Wed Jun 15 10:18:15 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:18:15.283624 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=100 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:18:15.284543 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9bc2f2-34ee-4140-8bc8-8d664485b054 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:18:15.389362 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646752cc-7c40-4f9f-9f37-578f45129ee4 None admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:18:15.407929 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9bc2f2-34ee-4140-8bc8-8d664485b054 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:18:15.422323 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:18:15.429392 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d22371-07a6-4db7-8ccc-e766c584f2e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:18:15.430344 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d22371-07a6-4db7-8ccc-e766c584f2e2 None admin] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:15.430959 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d22371-07a6-4db7-8ccc-e766c584f2e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:15.431563 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d22371-07a6-4db7-8ccc-e766c584f2e2 None admin] PATH_INFO: `/v3/groups` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:15.432483 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d22371-07a6-4db7-8ccc-e766c584f2e2 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:18:15.434181 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d22371-07a6-4db7-8ccc-e766c584f2e2 None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:18:15.450701 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4512c6-33b7-4646-97a1-5f80a63256ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:18:15.451618 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4512c6-33b7-4646-97a1-5f80a63256ad None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:15.451902 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4512c6-33b7-4646-97a1-5f80a63256ad None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:15.452230 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4512c6-33b7-4646-97a1-5f80a63256ad None admin] PATH_INFO: `/v3/users/948979771ffa47d184a627149510a973` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:15.452969 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 154/308] 10.222.0.49 () {60 vars in 1134 bytes} [Wed Jun 15 10:18:15 2022] POST /identity/v3/groups => generated 229 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 10:18:15.474118 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb4512c6-33b7-4646-97a1-5f80a63256ad None admin] RBAC: Authorizing `identity:get_user(user_id=948979771ffa47d184a627149510a973)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:18:15.475738 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb4512c6-33b7-4646-97a1-5f80a63256ad None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:18:15.483009 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 155/309] 10.222.0.49 () {56 vars in 1177 bytes} [Wed Jun 15 10:18:15 2022] GET /identity/v3/users/948979771ffa47d184a627149510a973 => generated 293 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:18:15.484536 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d22371-07a6-4db7-8ccc-e766c584f2e2 None admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:18:15.561604 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:18:15.613910 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d22371-07a6-4db7-8ccc-e766c584f2e2 None admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:18:15.652516 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa4b5b0-a5d6-45d1-b1ba-ca687bc96d9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:18:15.654091 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa4b5b0-a5d6-45d1-b1ba-ca687bc96d9a None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:15.654593 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa4b5b0-a5d6-45d1-b1ba-ca687bc96d9a None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:15.655063 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa4b5b0-a5d6-45d1-b1ba-ca687bc96d9a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:15.656123 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa4b5b0-a5d6-45d1-b1ba-ca687bc96d9a None admin] RBAC: Adding query filter params (role.id=301a3c8a23ba40edbc35df1da8af0de9, user.id=948979771ffa47d184a627149510a973, scope.system=all) {{(pid=80119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:18:15.656634 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa4b5b0-a5d6-45d1-b1ba-ca687bc96d9a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:18:15.658445 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa4b5b0-a5d6-45d1-b1ba-ca687bc96d9a None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:18:15.664385 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 155/310] 10.222.0.49 () {56 vars in 1308 bytes} [Wed Jun 15 10:18:15 2022] GET /identity/v3/role_assignments?user.id=948979771ffa47d184a627149510a973&scope.system=all&role.id=301a3c8a23ba40edbc35df1da8af0de9 => generated 226 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:18:16.135808 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f365f01-bcb6-4b9f-9df8-27c33adba0cf None None] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:16.136239 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f365f01-bcb6-4b9f-9df8-27c33adba0cf None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:16.136411 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f365f01-bcb6-4b9f-9df8-27c33adba0cf None None] PATH_INFO: `/` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:16.138038 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 156/311] 10.222.0.49 () {54 vars in 926 bytes} [Wed Jun 15 10:18:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:18:16.143292 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd18e5f-1449-4fa2-8704-7464e8e4bd87 None None] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:16.143590 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd18e5f-1449-4fa2-8704-7464e8e4bd87 None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:16.144033 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd18e5f-1449-4fa2-8704-7464e8e4bd87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:16.160867 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.auth.core [None req-6fd18e5f-1449-4fa2-8704-7464e8e4bd87 None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:18:16.162071 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-6fd18e5f-1449-4fa2-8704-7464e8e4bd87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:18:16.239199 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 156/312] 10.222.0.49 () {58 vars in 1023 bytes} [Wed Jun 15 10:18:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 10:18:16.255793 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f365f01-bcb6-4b9f-9df8-27c33adba0cf None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:18:16.364883 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f365f01-bcb6-4b9f-9df8-27c33adba0cf None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:18:16.407326 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e66fd8-eb7a-41bd-954e-61ad8fcba803 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:18:16.408109 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e66fd8-eb7a-41bd-954e-61ad8fcba803 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:16.408635 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e66fd8-eb7a-41bd-954e-61ad8fcba803 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:16.408635 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e66fd8-eb7a-41bd-954e-61ad8fcba803 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:16.426695 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e66fd8-eb7a-41bd-954e-61ad8fcba803 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 10:18:16.434847 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application [None req-23e66fd8-eb7a-41bd-954e-61ad8fcba803 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 15 10:18:16.434847 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:18:16.434847 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:18:16.434847 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:18:16.434847 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:18:16.434847 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:18:16.434847 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:18:16.434847 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:18:16.434847 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:18:16.434847 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:18:16.434847 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:18:16.434847 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:18:16.434847 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 10:18:16.434847 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 10:18:16.434847 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 10:18:16.434847 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 10:18:16.435793 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 10:18:16.435793 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 10:18:16.435793 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 10:18:16.435793 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 10:18:16.435793 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:18:16.435793 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:18:16.435793 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:18:16.435793 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:18:16.435793 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:18:16.435793 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:18:16.435793 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:18:16.435793 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application with Lock( Jun 15 10:18:16.435793 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:18:16.435793 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:18:16.435793 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:18:16.435793 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:18:16.436603 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:18:16.436603 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:18:16.436603 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:18:16.436603 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:18:16.436603 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 10:18:16.436603 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 10:18:16.436603 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 10:18:16.436603 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 10:18:16.436603 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 10:18:16.436603 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 10:18:16.436603 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 15 10:18:16.436603 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Jun 15 10:18:16.440890 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 157/313] 10.222.0.49 () {56 vars in 1099 bytes} [Wed Jun 15 10:18:16 2022] GET /identity/v3/roles/reader => generated 84 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:18:16.442304 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd18e5f-1449-4fa2-8704-7464e8e4bd87 None None] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:18:16.501142 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=100 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:18:16.544731 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd18e5f-1449-4fa2-8704-7464e8e4bd87 None None] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:18:16.577707 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ef16b0-1c5b-4ec9-bf0a-310013b5a20c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:18:16.578376 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ef16b0-1c5b-4ec9-bf0a-310013b5a20c None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:16.578660 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ef16b0-1c5b-4ec9-bf0a-310013b5a20c None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:16.578933 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ef16b0-1c5b-4ec9-bf0a-310013b5a20c None admin] PATH_INFO: `/v3/roles` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:16.579694 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ef16b0-1c5b-4ec9-bf0a-310013b5a20c None admin] RBAC: Adding query filter params (name=reader) {{(pid=80119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:18:16.580001 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ef16b0-1c5b-4ec9-bf0a-310013b5a20c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:18:16.581382 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ef16b0-1c5b-4ec9-bf0a-310013b5a20c None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:18:16.594586 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 157/314] 10.222.0.49 () {56 vars in 1101 bytes} [Wed Jun 15 10:18:16 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:18:16.595937 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e66fd8-eb7a-41bd-954e-61ad8fcba803 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:18:16.626308 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:18:16.705831 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e66fd8-eb7a-41bd-954e-61ad8fcba803 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:18:16.741716 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1ad8b8-51fc-469f-be6d-27d46993e9d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:18:16.742629 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1ad8b8-51fc-469f-be6d-27d46993e9d9 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:16.742910 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1ad8b8-51fc-469f-be6d-27d46993e9d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:16.743207 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1ad8b8-51fc-469f-be6d-27d46993e9d9 None admin] PATH_INFO: `/v3/users/b142a76f3f994d00b76f1eecb82a7541` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:16.744526 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e1ad8b8-51fc-469f-be6d-27d46993e9d9 None admin] RBAC: Authorizing `identity:get_user(user_id=b142a76f3f994d00b76f1eecb82a7541)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:18:16.746028 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e1ad8b8-51fc-469f-be6d-27d46993e9d9 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:18:16.752388 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 158/315] 10.222.0.49 () {56 vars in 1177 bytes} [Wed Jun 15 10:18:16 2022] GET /identity/v3/users/b142a76f3f994d00b76f1eecb82a7541 => generated 297 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:18:16.753837 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ef16b0-1c5b-4ec9-bf0a-310013b5a20c None admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:18:16.865114 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ef16b0-1c5b-4ec9-bf0a-310013b5a20c None admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:18:16.888792 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=100 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:18:16.900654 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975d7887-6a8b-4cd8-bc71-543c91c6a612 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:18:16.901371 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975d7887-6a8b-4cd8-bc71-543c91c6a612 None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:16.902024 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975d7887-6a8b-4cd8-bc71-543c91c6a612 None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:16.902966 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975d7887-6a8b-4cd8-bc71-543c91c6a612 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:16.972903 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975d7887-6a8b-4cd8-bc71-543c91c6a612 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:18:16.974295 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975d7887-6a8b-4cd8-bc71-543c91c6a612 None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:18:17.074246 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 158/316] 10.222.0.49 () {58 vars in 1282 bytes} [Wed Jun 15 10:18:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 15 10:18:17.080231 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1ad8b8-51fc-469f-be6d-27d46993e9d9 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:18:17.180117 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:18:17.187238 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1ad8b8-51fc-469f-be6d-27d46993e9d9 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:18:17.223229 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddbd274a-4cd6-4ac6-ac02-b39ac9cbacc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:18:17.224032 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbd274a-4cd6-4ac6-ac02-b39ac9cbacc5 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:17.224321 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbd274a-4cd6-4ac6-ac02-b39ac9cbacc5 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:17.224596 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbd274a-4cd6-4ac6-ac02-b39ac9cbacc5 None admin] PATH_INFO: `/v3/projects/df1f045b58c24b4dba0964eff7d2e540` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:17.225824 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddbd274a-4cd6-4ac6-ac02-b39ac9cbacc5 None admin] RBAC: Authorizing `identity:get_project(project_id=df1f045b58c24b4dba0964eff7d2e540)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:18:17.227314 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddbd274a-4cd6-4ac6-ac02-b39ac9cbacc5 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:18:17.229210 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 159/317] 10.222.0.49 () {56 vars in 1186 bytes} [Wed Jun 15 10:18:17 2022] GET /identity/v3/projects/df1f045b58c24b4dba0964eff7d2e540 => generated 302 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:18:17.236200 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975d7887-6a8b-4cd8-bc71-543c91c6a612 None admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:18:17.264790 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=100 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:18:17.318195 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ad5329-a8e7-4c08-aa07-174bc9961a46 None None] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:17.318504 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ad5329-a8e7-4c08-aa07-174bc9961a46 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:17.318768 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ad5329-a8e7-4c08-aa07-174bc9961a46 None None] PATH_INFO: `/` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:17.320453 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 160/318] 10.222.0.49 () {54 vars in 926 bytes} [Wed Jun 15 10:18:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 15 10:18:17.326096 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb79e142-975d-4b45-b4c2-b8f224dfa862 None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:17.326392 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb79e142-975d-4b45-b4c2-b8f224dfa862 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:17.326678 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb79e142-975d-4b45-b4c2-b8f224dfa862 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:17.347181 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-eb79e142-975d-4b45-b4c2-b8f224dfa862 None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:18:17.349002 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-eb79e142-975d-4b45-b4c2-b8f224dfa862 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:18:17.350778 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975d7887-6a8b-4cd8-bc71-543c91c6a612 None admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:18:17.392892 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f64282-7c9f-4054-9c92-dc508e6f26bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:18:17.393562 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f64282-7c9f-4054-9c92-dc508e6f26bd None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:17.393850 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f64282-7c9f-4054-9c92-dc508e6f26bd None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:17.394291 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f64282-7c9f-4054-9c92-dc508e6f26bd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:17.395187 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f64282-7c9f-4054-9c92-dc508e6f26bd None admin] RBAC: Adding query filter params (role.id=301a3c8a23ba40edbc35df1da8af0de9, scope.project.id=df1f045b58c24b4dba0964eff7d2e540, user.id=b142a76f3f994d00b76f1eecb82a7541) {{(pid=80119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:18:17.396301 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f64282-7c9f-4054-9c92-dc508e6f26bd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:18:17.397778 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f64282-7c9f-4054-9c92-dc508e6f26bd None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:18:17.405505 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 159/319] 10.222.0.49 () {56 vars in 1374 bytes} [Wed Jun 15 10:18:17 2022] GET /identity/v3/role_assignments?user.id=b142a76f3f994d00b76f1eecb82a7541&scope.project.id=df1f045b58c24b4dba0964eff7d2e540&role.id=301a3c8a23ba40edbc35df1da8af0de9 => generated 608 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:18:17.434404 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 161/320] 10.222.0.49 () {58 vars in 1023 bytes} [Wed Jun 15 10:18:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 15 10:18:17.458324 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f64282-7c9f-4054-9c92-dc508e6f26bd None admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:18:17.557359 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=100 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:18:17.572923 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f64282-7c9f-4054-9c92-dc508e6f26bd None admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:18:17.609051 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecda366d-ff84-41d5-9d18-889c89a98c24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:18:17.609869 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecda366d-ff84-41d5-9d18-889c89a98c24 None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:17.610158 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecda366d-ff84-41d5-9d18-889c89a98c24 None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:17.610438 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecda366d-ff84-41d5-9d18-889c89a98c24 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:17.621349 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecda366d-ff84-41d5-9d18-889c89a98c24 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 10:18:17.629219 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application [None req-ecda366d-ff84-41d5-9d18-889c89a98c24 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 15 10:18:17.629219 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:18:17.629219 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:18:17.629219 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:18:17.629219 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:18:17.629219 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:18:17.629219 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:18:17.629219 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:18:17.629219 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:18:17.629219 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:18:17.629219 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:18:17.629219 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:18:17.629219 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 10:18:17.629219 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 10:18:17.629219 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 10:18:17.629219 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 10:18:17.630098 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 10:18:17.630098 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 10:18:17.630098 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 10:18:17.630098 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 10:18:17.630098 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:18:17.630098 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:18:17.630098 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:18:17.630098 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:18:17.630098 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:18:17.630098 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:18:17.630098 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:18:17.630098 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application with Lock( Jun 15 10:18:17.630098 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:18:17.630098 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:18:17.630098 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:18:17.630098 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:18:17.630910 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:18:17.630910 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:18:17.630910 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:18:17.630910 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:18:17.630910 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 10:18:17.630910 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 10:18:17.630910 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 10:18:17.630910 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 10:18:17.630910 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 10:18:17.630910 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 10:18:17.630910 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 15 10:18:17.630910 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application Jun 15 10:18:17.630910 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 160/321] 10.222.0.49 () {56 vars in 1099 bytes} [Wed Jun 15 10:18:17 2022] GET /identity/v3/roles/member => generated 84 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:18:17.636791 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb79e142-975d-4b45-b4c2-b8f224dfa862 None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:18:17.741149 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:18:17.749328 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb79e142-975d-4b45-b4c2-b8f224dfa862 None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:18:17.793677 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ffa1e7-2225-4d3d-90e9-6a58394e460b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:18:17.794127 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ffa1e7-2225-4d3d-90e9-6a58394e460b None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:17.794413 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ffa1e7-2225-4d3d-90e9-6a58394e460b None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:17.794692 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ffa1e7-2225-4d3d-90e9-6a58394e460b None admin] PATH_INFO: `/v3/roles` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:17.795517 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ffa1e7-2225-4d3d-90e9-6a58394e460b None admin] RBAC: Adding query filter params (name=member) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:18:17.795829 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ffa1e7-2225-4d3d-90e9-6a58394e460b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:18:17.797275 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ffa1e7-2225-4d3d-90e9-6a58394e460b None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:18:17.801172 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6c97d4-2651-4b33-93fe-25fd10be4ed7 None None] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:17.801462 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6c97d4-2651-4b33-93fe-25fd10be4ed7 None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:17.801743 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6c97d4-2651-4b33-93fe-25fd10be4ed7 None None] PATH_INFO: `/` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:17.803599 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 161/322] 10.222.0.49 () {54 vars in 926 bytes} [Wed Jun 15 10:18:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:18:17.807677 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 162/323] 10.222.0.49 () {56 vars in 1101 bytes} [Wed Jun 15 10:18:17 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:18:17.808965 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8402838e-6dec-45c6-8ab8-1ddaa2637097 None None] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:17.809256 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8402838e-6dec-45c6-8ab8-1ddaa2637097 None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:17.809535 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8402838e-6dec-45c6-8ab8-1ddaa2637097 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:17.813738 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ffa1e7-2225-4d3d-90e9-6a58394e460b None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:18:17.828669 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.auth.core [None req-8402838e-6dec-45c6-8ab8-1ddaa2637097 None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:18:17.829865 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-8402838e-6dec-45c6-8ab8-1ddaa2637097 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:18:17.897634 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 162/324] 10.222.0.49 () {58 vars in 1023 bytes} [Wed Jun 15 10:18:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 15 10:18:17.899113 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187dbe30-ff96-48ad-8ded-5719c93355e7 None None] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:17.899419 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187dbe30-ff96-48ad-8ded-5719c93355e7 None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:17.899702 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187dbe30-ff96-48ad-8ded-5719c93355e7 None None] PATH_INFO: `/` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:17.901244 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 163/325] 10.222.0.49 () {54 vars in 926 bytes} [Wed Jun 15 10:18:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:18:17.906645 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7e2755-e944-4c0c-bfc3-c3b1fb5d8afb None None] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:17.906942 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7e2755-e944-4c0c-bfc3-c3b1fb5d8afb None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:17.907269 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7e2755-e944-4c0c-bfc3-c3b1fb5d8afb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:17.923957 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.auth.core [None req-ba7e2755-e944-4c0c-bfc3-c3b1fb5d8afb None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:18:17.925113 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ffa1e7-2225-4d3d-90e9-6a58394e460b None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:18:17.925222 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-ba7e2755-e944-4c0c-bfc3-c3b1fb5d8afb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:18:17.958818 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6865ca-69ed-471f-8fe7-94479036650c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:18:17.959719 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6865ca-69ed-471f-8fe7-94479036650c None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:17.960002 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6865ca-69ed-471f-8fe7-94479036650c None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:17.960271 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6865ca-69ed-471f-8fe7-94479036650c None admin] PATH_INFO: `/v3/users/9529db0ddca44678bd92a22b908cba6f` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:17.961455 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f6865ca-69ed-471f-8fe7-94479036650c None admin] RBAC: Authorizing `identity:get_user(user_id=9529db0ddca44678bd92a22b908cba6f)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:18:17.962875 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f6865ca-69ed-471f-8fe7-94479036650c None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:18:17.964525 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 163/326] 10.222.0.49 () {56 vars in 1177 bytes} [Wed Jun 15 10:18:17 2022] GET /identity/v3/users/9529db0ddca44678bd92a22b908cba6f => generated 293 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 10:18:17.965272 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6865ca-69ed-471f-8fe7-94479036650c None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:18:17.990106 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:18:17.996324 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 164/327] 10.222.0.49 () {58 vars in 1023 bytes} [Wed Jun 15 10:18:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 15 10:18:17.997206 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7e2755-e944-4c0c-bfc3-c3b1fb5d8afb None None] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:18:18.005240 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=100 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:18:18.064910 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6865ca-69ed-471f-8fe7-94479036650c None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:18:18.095717 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7e2755-e944-4c0c-bfc3-c3b1fb5d8afb None None] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:18:18.097764 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dee4f8a-f5ce-4159-87e7-77a3327a18df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:18:18.098719 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dee4f8a-f5ce-4159-87e7-77a3327a18df None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:18.099000 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dee4f8a-f5ce-4159-87e7-77a3327a18df None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:18.099309 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dee4f8a-f5ce-4159-87e7-77a3327a18df None admin] PATH_INFO: `/v3/domains/default` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:18.100469 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dee4f8a-f5ce-4159-87e7-77a3327a18df None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:18:18.101899 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dee4f8a-f5ce-4159-87e7-77a3327a18df None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:18:18.103535 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 164/328] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Jun 15 10:18:17 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:18:18.104288 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dee4f8a-f5ce-4159-87e7-77a3327a18df None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:18:18.130102 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f2c4c9-8f11-4f99-8ca9-69c0fdb741b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:18:18.130763 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f2c4c9-8f11-4f99-8ca9-69c0fdb741b5 None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:18.131051 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f2c4c9-8f11-4f99-8ca9-69c0fdb741b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:18.131367 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f2c4c9-8f11-4f99-8ca9-69c0fdb741b5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:18.132182 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f2c4c9-8f11-4f99-8ca9-69c0fdb741b5 None admin] RBAC: Adding query filter params (role.id=6876e57cc51c448097ab109d35b2a3da, user.id=9529db0ddca44678bd92a22b908cba6f, scope.system=all) {{(pid=80119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:18:18.132491 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f2c4c9-8f11-4f99-8ca9-69c0fdb741b5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:18:18.133923 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f2c4c9-8f11-4f99-8ca9-69c0fdb741b5 None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:18:18.145829 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 165/329] 10.222.0.49 () {56 vars in 1308 bytes} [Wed Jun 15 10:18:17 2022] GET /identity/v3/role_assignments?user.id=9529db0ddca44678bd92a22b908cba6f&scope.system=all&role.id=6876e57cc51c448097ab109d35b2a3da => generated 510 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:18:18.146594 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f2c4c9-8f11-4f99-8ca9-69c0fdb741b5 None admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:18:18.209634 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:18:18.221096 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dee4f8a-f5ce-4159-87e7-77a3327a18df None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:18:18.226233 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=100 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:18:18.260045 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541649c8-7c4a-48d5-a9ab-963b39c6585a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:18:18.262025 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541649c8-7c4a-48d5-a9ab-963b39c6585a None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:18.262600 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541649c8-7c4a-48d5-a9ab-963b39c6585a None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:18.263621 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541649c8-7c4a-48d5-a9ab-963b39c6585a None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:18.271775 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f2c4c9-8f11-4f99-8ca9-69c0fdb741b5 None admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:18:18.275696 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-541649c8-7c4a-48d5-a9ab-963b39c6585a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 10:18:18.284195 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application [None req-541649c8-7c4a-48d5-a9ab-963b39c6585a None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 15 10:18:18.284195 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:18:18.284195 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:18:18.284195 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:18:18.284195 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:18:18.284195 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:18:18.284195 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:18:18.284195 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:18:18.284195 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:18:18.284195 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:18:18.284195 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:18:18.284195 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:18:18.284195 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 10:18:18.284195 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 10:18:18.284195 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 10:18:18.284195 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 10:18:18.285165 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 10:18:18.285165 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 10:18:18.285165 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 10:18:18.285165 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 10:18:18.285165 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:18:18.285165 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:18:18.285165 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:18:18.285165 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:18:18.285165 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:18:18.285165 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:18:18.285165 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:18:18.285165 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application with Lock( Jun 15 10:18:18.285165 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:18:18.285165 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:18:18.285165 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:18:18.285165 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:18:18.285967 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:18:18.285967 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:18:18.285967 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:18:18.285967 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:18:18.285967 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 10:18:18.285967 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 10:18:18.285967 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 10:18:18.285967 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 10:18:18.285967 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 10:18:18.285967 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 10:18:18.285967 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 15 10:18:18.285967 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Jun 15 10:18:18.285967 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 165/330] 10.222.0.49 () {56 vars in 1099 bytes} [Wed Jun 15 10:18:18 2022] GET /identity/v3/roles/reader => generated 84 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:18:18.291581 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541649c8-7c4a-48d5-a9ab-963b39c6585a None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:18:18.315125 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b61d228-6833-474e-b06c-3e82644f30fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:18:18.315727 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b61d228-6833-474e-b06c-3e82644f30fb None admin] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:18.316010 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b61d228-6833-474e-b06c-3e82644f30fb None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:18.316291 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b61d228-6833-474e-b06c-3e82644f30fb None admin] PATH_INFO: `/v3/groups` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:18.316975 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b61d228-6833-474e-b06c-3e82644f30fb None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:18:18.318413 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b61d228-6833-474e-b06c-3e82644f30fb None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:18:18.335161 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 166/331] 10.222.0.49 () {60 vars in 1134 bytes} [Wed Jun 15 10:18:18 2022] POST /identity/v3/groups => generated 226 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 10:18:18.405930 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541649c8-7c4a-48d5-a9ab-963b39c6585a None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:18:18.442006 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25c40ab-585f-4937-b127-6bab8deacb45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:18:18.442699 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25c40ab-585f-4937-b127-6bab8deacb45 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:18.442984 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25c40ab-585f-4937-b127-6bab8deacb45 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:18.443303 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25c40ab-585f-4937-b127-6bab8deacb45 None admin] PATH_INFO: `/v3/roles` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:18.444069 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c25c40ab-585f-4937-b127-6bab8deacb45 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:18:18.444385 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c25c40ab-585f-4937-b127-6bab8deacb45 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:18:18.445800 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c25c40ab-585f-4937-b127-6bab8deacb45 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:18:18.455542 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 166/332] 10.222.0.49 () {56 vars in 1101 bytes} [Wed Jun 15 10:18:18 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 10:18:18.461220 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b61d228-6833-474e-b06c-3e82644f30fb None admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:18:18.565698 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b61d228-6833-474e-b06c-3e82644f30fb None admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:18:18.591207 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:18:18.604102 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db63ceb-334d-408e-b1c8-b58c536e5eec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:18:18.605002 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db63ceb-334d-408e-b1c8-b58c536e5eec None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:18.605289 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db63ceb-334d-408e-b1c8-b58c536e5eec None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:18.605569 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db63ceb-334d-408e-b1c8-b58c536e5eec None admin] PATH_INFO: `/v3/users/948979771ffa47d184a627149510a973` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:18.606827 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db63ceb-334d-408e-b1c8-b58c536e5eec None admin] RBAC: Authorizing `identity:get_user(user_id=948979771ffa47d184a627149510a973)` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:18:18.608339 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db63ceb-334d-408e-b1c8-b58c536e5eec None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:18:18.610362 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 167/333] 10.222.0.49 () {56 vars in 1177 bytes} [Wed Jun 15 10:18:18 2022] GET /identity/v3/users/948979771ffa47d184a627149510a973 => generated 293 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:18:18.615650 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25c40ab-585f-4937-b127-6bab8deacb45 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:18:18.634831 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:18:18.728224 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25c40ab-585f-4937-b127-6bab8deacb45 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:18:18.763417 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189f77cf-dee3-4b2f-9a38-b32987769280 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:18:18.764294 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189f77cf-dee3-4b2f-9a38-b32987769280 None admin] REQUEST_METHOD: `PUT` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:18.764581 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189f77cf-dee3-4b2f-9a38-b32987769280 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:18.764859 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189f77cf-dee3-4b2f-9a38-b32987769280 None admin] PATH_INFO: `/v3/system/users/948979771ffa47d184a627149510a973/roles/301a3c8a23ba40edbc35df1da8af0de9` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:18.766360 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-189f77cf-dee3-4b2f-9a38-b32987769280 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=948979771ffa47d184a627149510a973, role_id=301a3c8a23ba40edbc35df1da8af0de9)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:18:18.767815 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-189f77cf-dee3-4b2f-9a38-b32987769280 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:18:18.776936 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 167/334] 10.222.0.49 () {58 vars in 1334 bytes} [Wed Jun 15 10:18:18 2022] PUT /identity/v3/system/users/948979771ffa47d184a627149510a973/roles/301a3c8a23ba40edbc35df1da8af0de9 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:18:20.698811 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88db99f3-5d7a-483f-a55a-c2a2ec95c807 None None] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:20.699448 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88db99f3-5d7a-483f-a55a-c2a2ec95c807 None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:20.699448 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88db99f3-5d7a-483f-a55a-c2a2ec95c807 None None] PATH_INFO: `/` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:20.702553 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 168/335] 10.222.0.49 () {54 vars in 926 bytes} [Wed Jun 15 10:18:20 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:18:20.707875 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08378232-be4b-4742-8245-63c34df508b2 None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:20.708291 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08378232-be4b-4742-8245-63c34df508b2 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:20.708477 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08378232-be4b-4742-8245-63c34df508b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:20.732192 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-08378232-be4b-4742-8245-63c34df508b2 None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:18:20.733415 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-08378232-be4b-4742-8245-63c34df508b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:18:20.821447 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 168/336] 10.222.0.49 () {58 vars in 1023 bytes} [Wed Jun 15 10:18:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 10:18:20.860591 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88db99f3-5d7a-483f-a55a-c2a2ec95c807 None None] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:18:20.976576 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88db99f3-5d7a-483f-a55a-c2a2ec95c807 None None] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:18:21.014665 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4119b9-b529-498e-9837-5653c76b1249 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:18:21.015545 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4119b9-b529-498e-9837-5653c76b1249 None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:21.015836 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4119b9-b529-498e-9837-5653c76b1249 None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:21.016114 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4119b9-b529-498e-9837-5653c76b1249 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:21.027683 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b4119b9-b529-498e-9837-5653c76b1249 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 10:18:21.035850 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application [None req-5b4119b9-b529-498e-9837-5653c76b1249 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 15 10:18:21.035850 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:18:21.035850 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:18:21.035850 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:18:21.035850 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:18:21.035850 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:18:21.035850 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:18:21.035850 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:18:21.035850 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:18:21.035850 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:18:21.035850 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:18:21.035850 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:18:21.035850 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 10:18:21.035850 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 10:18:21.035850 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 10:18:21.035850 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 10:18:21.036805 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 10:18:21.036805 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 10:18:21.036805 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 10:18:21.036805 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 10:18:21.036805 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:18:21.036805 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:18:21.036805 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:18:21.036805 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:18:21.036805 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:18:21.036805 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:18:21.036805 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:18:21.036805 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application with Lock( Jun 15 10:18:21.036805 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:18:21.036805 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:18:21.036805 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:18:21.036805 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:18:21.037580 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:18:21.037580 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:18:21.037580 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:18:21.037580 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:18:21.037580 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 10:18:21.037580 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 10:18:21.037580 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 10:18:21.037580 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 10:18:21.037580 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 10:18:21.037580 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 10:18:21.037580 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 15 10:18:21.037580 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application Jun 15 10:18:21.037580 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 169/337] 10.222.0.49 () {56 vars in 1099 bytes} [Wed Jun 15 10:18:20 2022] GET /identity/v3/roles/member => generated 84 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:18:21.043258 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08378232-be4b-4742-8245-63c34df508b2 None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:18:21.053809 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:18:21.156596 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08378232-be4b-4742-8245-63c34df508b2 None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:18:21.204268 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da55fa9-4ab0-450e-8a1e-861d79e595d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:18:21.204955 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da55fa9-4ab0-450e-8a1e-861d79e595d6 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:21.205243 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da55fa9-4ab0-450e-8a1e-861d79e595d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:21.205520 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da55fa9-4ab0-450e-8a1e-861d79e595d6 None admin] PATH_INFO: `/v3/roles` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:21.206286 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da55fa9-4ab0-450e-8a1e-861d79e595d6 None admin] RBAC: Adding query filter params (name=member) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:18:21.206595 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da55fa9-4ab0-450e-8a1e-861d79e595d6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:18:21.208069 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da55fa9-4ab0-450e-8a1e-861d79e595d6 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:18:21.223476 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 169/338] 10.222.0.49 () {56 vars in 1101 bytes} [Wed Jun 15 10:18:21 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 10:18:21.228805 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4119b9-b529-498e-9837-5653c76b1249 None admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:18:21.291501 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=100 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:18:21.314368 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fb2319-332e-49d8-9878-08ed1f8ea848 None None] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:21.314676 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fb2319-332e-49d8-9878-08ed1f8ea848 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:21.314938 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fb2319-332e-49d8-9878-08ed1f8ea848 None None] PATH_INFO: `/` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:21.316349 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 170/339] 10.222.0.49 () {54 vars in 926 bytes} [Wed Jun 15 10:18:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:18:21.321573 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3f0866-93e7-4e8a-ba57-b74a48bf8ab2 None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:21.322036 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3f0866-93e7-4e8a-ba57-b74a48bf8ab2 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:21.322326 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3f0866-93e7-4e8a-ba57-b74a48bf8ab2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:21.348040 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-9c3f0866-93e7-4e8a-ba57-b74a48bf8ab2 None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:18:21.349601 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-9c3f0866-93e7-4e8a-ba57-b74a48bf8ab2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:18:21.374856 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4119b9-b529-498e-9837-5653c76b1249 None admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:18:21.413816 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e465d997-bc02-4fb4-ae75-243cd78de970 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:18:21.414817 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e465d997-bc02-4fb4-ae75-243cd78de970 None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:21.415379 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e465d997-bc02-4fb4-ae75-243cd78de970 None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:21.415813 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e465d997-bc02-4fb4-ae75-243cd78de970 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:21.479879 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:18:21.491832 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 171/340] 10.222.0.49 () {58 vars in 1023 bytes} [Wed Jun 15 10:18:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 15 10:18:21.526070 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e465d997-bc02-4fb4-ae75-243cd78de970 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:18:21.527979 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e465d997-bc02-4fb4-ae75-243cd78de970 None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:18:21.540677 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3f0866-93e7-4e8a-ba57-b74a48bf8ab2 None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:18:21.632999 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=100 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:18:21.698524 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 170/341] 10.222.0.49 () {58 vars in 1282 bytes} [Wed Jun 15 10:18:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 470 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 15 10:18:21.706191 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e465d997-bc02-4fb4-ae75-243cd78de970 None admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:18:21.733648 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3f0866-93e7-4e8a-ba57-b74a48bf8ab2 None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:18:21.784163 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04cd9702-4f25-49df-9bd3-ef1d8a4d0917 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:18:21.785240 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04cd9702-4f25-49df-9bd3-ef1d8a4d0917 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:21.785678 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04cd9702-4f25-49df-9bd3-ef1d8a4d0917 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:21.786099 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04cd9702-4f25-49df-9bd3-ef1d8a4d0917 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:21.803915 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04cd9702-4f25-49df-9bd3-ef1d8a4d0917 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 10:18:21.812047 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application [None req-04cd9702-4f25-49df-9bd3-ef1d8a4d0917 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 15 10:18:21.812047 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:18:21.812047 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:18:21.812047 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:18:21.812047 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:18:21.812047 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:18:21.812047 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:18:21.812047 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:18:21.812047 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:18:21.812047 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:18:21.812047 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:18:21.812047 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:18:21.812047 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 10:18:21.812047 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 10:18:21.812047 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 10:18:21.812047 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 10:18:21.813154 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 10:18:21.813154 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 10:18:21.813154 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 10:18:21.813154 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 10:18:21.813154 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:18:21.813154 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:18:21.813154 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:18:21.813154 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:18:21.813154 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:18:21.813154 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:18:21.813154 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:18:21.813154 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application with Lock( Jun 15 10:18:21.813154 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:18:21.813154 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:18:21.813154 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:18:21.813154 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:18:21.814186 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:18:21.814186 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:18:21.814186 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:18:21.814186 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:18:21.814186 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 10:18:21.814186 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 10:18:21.814186 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 10:18:21.814186 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 10:18:21.814186 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 10:18:21.814186 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 10:18:21.814186 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 15 10:18:21.814186 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Jun 15 10:18:21.816515 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 172/342] 10.222.0.49 () {56 vars in 1099 bytes} [Wed Jun 15 10:18:21 2022] GET /identity/v3/roles/member => generated 84 bytes in 276 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:18:21.821922 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04cd9702-4f25-49df-9bd3-ef1d8a4d0917 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:18:21.866739 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e465d997-bc02-4fb4-ae75-243cd78de970 None admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:18:21.885444 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:18:21.937476 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04cd9702-4f25-49df-9bd3-ef1d8a4d0917 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:18:21.955929 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6a81ef-0438-4bd9-8dcd-51229af3c015 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:18:21.957115 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6a81ef-0438-4bd9-8dcd-51229af3c015 None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:21.957554 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6a81ef-0438-4bd9-8dcd-51229af3c015 None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:21.957984 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6a81ef-0438-4bd9-8dcd-51229af3c015 None admin] PATH_INFO: `/v3/projects/5f3cd857995e418386bb22cdf90c1ae9` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:21.959488 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6a81ef-0438-4bd9-8dcd-51229af3c015 None admin] RBAC: Authorizing `identity:get_project(project_id=5f3cd857995e418386bb22cdf90c1ae9)` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:18:21.961119 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6a81ef-0438-4bd9-8dcd-51229af3c015 None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:18:21.964330 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 171/343] 10.222.0.49 () {56 vars in 1186 bytes} [Wed Jun 15 10:18:21 2022] GET /identity/v3/projects/5f3cd857995e418386bb22cdf90c1ae9 => generated 298 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:18:21.966319 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4f1539-0873-42f2-ad78-6810d6e028a7 None None] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:21.966769 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4f1539-0873-42f2-ad78-6810d6e028a7 None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:21.967221 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4f1539-0873-42f2-ad78-6810d6e028a7 None None] PATH_INFO: `/` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:21.973268 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 172/344] 10.222.0.49 () {54 vars in 926 bytes} [Wed Jun 15 10:18:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:18:21.976468 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588a3eab-540b-4ea9-af81-99d1fd818e6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:18:21.977334 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588a3eab-540b-4ea9-af81-99d1fd818e6b None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:21.977744 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aff2fe3-58bf-4ceb-b978-2647420f920f None None] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:21.977934 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588a3eab-540b-4ea9-af81-99d1fd818e6b None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:21.978046 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aff2fe3-58bf-4ceb-b978-2647420f920f None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:21.978304 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aff2fe3-58bf-4ceb-b978-2647420f920f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:21.978605 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588a3eab-540b-4ea9-af81-99d1fd818e6b None admin] PATH_INFO: `/v3/roles` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:21.979519 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-588a3eab-540b-4ea9-af81-99d1fd818e6b None admin] RBAC: Adding query filter params (name=member) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:18:21.979975 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-588a3eab-540b-4ea9-af81-99d1fd818e6b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:18:21.981525 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-588a3eab-540b-4ea9-af81-99d1fd818e6b None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:18:21.991350 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 173/345] 10.222.0.49 () {56 vars in 1101 bytes} [Wed Jun 15 10:18:21 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 10:18:21.992362 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588a3eab-540b-4ea9-af81-99d1fd818e6b None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:18:22.011591 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.auth.core [None req-7aff2fe3-58bf-4ceb-b978-2647420f920f None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:18:22.013915 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-7aff2fe3-58bf-4ceb-b978-2647420f920f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:18:22.030739 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=100 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:18:22.094324 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 173/346] 10.222.0.49 () {58 vars in 1023 bytes} [Wed Jun 15 10:18:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 15 10:18:22.095691 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aff2fe3-58bf-4ceb-b978-2647420f920f None None] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:18:22.111789 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588a3eab-540b-4ea9-af81-99d1fd818e6b None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:18:22.136577 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:18:22.151104 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b548fa-27e6-46c5-a617-f6dfc94dac35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:18:22.152586 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b548fa-27e6-46c5-a617-f6dfc94dac35 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:22.152871 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b548fa-27e6-46c5-a617-f6dfc94dac35 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:22.153213 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b548fa-27e6-46c5-a617-f6dfc94dac35 None admin] PATH_INFO: `/v3/groups/77182ac3056944e19934f133b3f7b1b3` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:22.161780 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b548fa-27e6-46c5-a617-f6dfc94dac35 None admin] RBAC: Authorizing `identity:get_group(group_id=77182ac3056944e19934f133b3f7b1b3)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:18:22.163295 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b548fa-27e6-46c5-a617-f6dfc94dac35 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:18:22.165217 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 174/347] 10.222.0.49 () {56 vars in 1180 bytes} [Wed Jun 15 10:18:21 2022] GET /identity/v3/groups/77182ac3056944e19934f133b3f7b1b3 => generated 226 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:18:22.166660 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61948940-709d-4fe9-b178-01daefdab1c4 None None] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:22.167157 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61948940-709d-4fe9-b178-01daefdab1c4 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:22.167737 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61948940-709d-4fe9-b178-01daefdab1c4 None None] PATH_INFO: `/` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:22.169801 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 175/348] 10.222.0.49 () {54 vars in 926 bytes} [Wed Jun 15 10:18:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:18:22.170681 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61948940-709d-4fe9-b178-01daefdab1c4 None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:18:22.216924 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aff2fe3-58bf-4ceb-b978-2647420f920f None None] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:18:22.257915 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2382b7f1-ba12-42a3-970a-d83482c85810 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:18:22.259746 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2382b7f1-ba12-42a3-970a-d83482c85810 None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:22.260267 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2382b7f1-ba12-42a3-970a-d83482c85810 None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:22.260723 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2382b7f1-ba12-42a3-970a-d83482c85810 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:22.281485 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=100 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:18:22.298445 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61948940-709d-4fe9-b178-01daefdab1c4 None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:18:22.338554 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2382b7f1-ba12-42a3-970a-d83482c85810 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:18:22.340497 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2382b7f1-ba12-42a3-970a-d83482c85810 None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:18:22.343115 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb22e47-304f-43d2-8c57-42888588dba2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:18:22.344310 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb22e47-304f-43d2-8c57-42888588dba2 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:22.344691 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb22e47-304f-43d2-8c57-42888588dba2 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:22.345047 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb22e47-304f-43d2-8c57-42888588dba2 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:22.361567 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb22e47-304f-43d2-8c57-42888588dba2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 10:18:22.371897 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application [None req-3fb22e47-304f-43d2-8c57-42888588dba2 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 15 10:18:22.371897 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:18:22.371897 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:18:22.371897 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:18:22.371897 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:18:22.371897 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:18:22.371897 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:18:22.371897 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:18:22.371897 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:18:22.371897 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:18:22.371897 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:18:22.371897 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:18:22.371897 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 10:18:22.371897 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 10:18:22.371897 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 10:18:22.371897 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 10:18:22.372876 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 10:18:22.372876 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 10:18:22.372876 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 10:18:22.372876 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 10:18:22.372876 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:18:22.372876 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:18:22.372876 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:18:22.372876 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:18:22.372876 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:18:22.372876 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:18:22.372876 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:18:22.372876 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application with Lock( Jun 15 10:18:22.372876 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:18:22.372876 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:18:22.372876 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:18:22.372876 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:18:22.373680 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:18:22.373680 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:18:22.373680 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:18:22.373680 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:18:22.373680 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 10:18:22.373680 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 10:18:22.373680 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 10:18:22.373680 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 10:18:22.373680 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 10:18:22.373680 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 10:18:22.373680 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 15 10:18:22.373680 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Jun 15 10:18:22.373680 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 176/349] 10.222.0.49 () {56 vars in 1114 bytes} [Wed Jun 15 10:18:22 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:18:22.374357 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb22e47-304f-43d2-8c57-42888588dba2 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:18:22.438419 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:18:22.468423 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 174/350] 10.222.0.49 () {58 vars in 1282 bytes} [Wed Jun 15 10:18:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 15 10:18:22.474023 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afad047-0626-445e-9e54-e1fd03d78e21 None None] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:22.474023 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afad047-0626-445e-9e54-e1fd03d78e21 None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:22.474023 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afad047-0626-445e-9e54-e1fd03d78e21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:22.490515 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.auth.core [None req-1afad047-0626-445e-9e54-e1fd03d78e21 None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:18:22.492290 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-1afad047-0626-445e-9e54-e1fd03d78e21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:18:22.521404 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb22e47-304f-43d2-8c57-42888588dba2 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:18:22.557889 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=100 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:18:22.566023 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17229e11-2104-4668-9fe1-0104e7996638 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:18:22.567164 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17229e11-2104-4668-9fe1-0104e7996638 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:22.567662 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17229e11-2104-4668-9fe1-0104e7996638 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:22.568167 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17229e11-2104-4668-9fe1-0104e7996638 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:22.569216 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17229e11-2104-4668-9fe1-0104e7996638 None admin] RBAC: Adding query filter params (group.id=77182ac3056944e19934f133b3f7b1b3, role.id=6876e57cc51c448097ab109d35b2a3da, scope.project.id=5f3cd857995e418386bb22cdf90c1ae9) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:18:22.569525 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17229e11-2104-4668-9fe1-0104e7996638 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:18:22.570951 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17229e11-2104-4668-9fe1-0104e7996638 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:18:22.578665 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 177/351] 10.222.0.49 () {56 vars in 1376 bytes} [Wed Jun 15 10:18:22 2022] GET /identity/v3/role_assignments?group.id=77182ac3056944e19934f133b3f7b1b3&scope.project.id=5f3cd857995e418386bb22cdf90c1ae9&role.id=6876e57cc51c448097ab109d35b2a3da => generated 260 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:18:22.579527 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17229e11-2104-4668-9fe1-0104e7996638 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:18:22.588678 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 175/352] 10.222.0.49 () {58 vars in 1023 bytes} [Wed Jun 15 10:18:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 10:18:22.591773 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89899a3-8c5d-418d-a58b-c8f12fd361cd None None] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:22.592408 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89899a3-8c5d-418d-a58b-c8f12fd361cd None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:22.592890 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89899a3-8c5d-418d-a58b-c8f12fd361cd None None] PATH_INFO: `/` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:22.594611 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 176/353] 10.222.0.49 () {54 vars in 926 bytes} [Wed Jun 15 10:18:22 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:18:22.595706 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89899a3-8c5d-418d-a58b-c8f12fd361cd None None] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:18:22.718571 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89899a3-8c5d-418d-a58b-c8f12fd361cd None None] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:18:22.729493 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17229e11-2104-4668-9fe1-0104e7996638 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:18:22.742330 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:18:22.762247 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfee43af-2263-4151-8f8f-a32b46375b18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:18:22.764322 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfee43af-2263-4151-8f8f-a32b46375b18 None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:22.764916 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfee43af-2263-4151-8f8f-a32b46375b18 None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:22.765453 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfee43af-2263-4151-8f8f-a32b46375b18 None admin] PATH_INFO: `/v3/projects/df1f045b58c24b4dba0964eff7d2e540` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:22.766910 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfee43af-2263-4151-8f8f-a32b46375b18 None admin] RBAC: Authorizing `identity:get_project(project_id=df1f045b58c24b4dba0964eff7d2e540)` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:18:22.768615 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfee43af-2263-4151-8f8f-a32b46375b18 None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:18:22.769551 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced1292d-5761-4a0d-89ca-333cedd7d055 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:18:22.770660 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced1292d-5761-4a0d-89ca-333cedd7d055 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:22.770660 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced1292d-5761-4a0d-89ca-333cedd7d055 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:22.770835 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced1292d-5761-4a0d-89ca-333cedd7d055 None admin] PATH_INFO: `/v3/roles` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:22.771856 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 177/354] 10.222.0.49 () {56 vars in 1186 bytes} [Wed Jun 15 10:18:22 2022] GET /identity/v3/projects/df1f045b58c24b4dba0964eff7d2e540 => generated 302 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:18:22.772665 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced1292d-5761-4a0d-89ca-333cedd7d055 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:18:22.772985 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced1292d-5761-4a0d-89ca-333cedd7d055 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:18:22.773806 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a3b25e-0e3e-43bf-bd72-bea968a5ab99 None None] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:22.774316 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a3b25e-0e3e-43bf-bd72-bea968a5ab99 None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:22.774417 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced1292d-5761-4a0d-89ca-333cedd7d055 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:18:22.774902 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a3b25e-0e3e-43bf-bd72-bea968a5ab99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:22.785631 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 178/355] 10.222.0.49 () {56 vars in 1111 bytes} [Wed Jun 15 10:18:22 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:18:22.786500 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced1292d-5761-4a0d-89ca-333cedd7d055 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:18:22.796096 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.auth.core [None req-c9a3b25e-0e3e-43bf-bd72-bea968a5ab99 None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:18:22.797645 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-c9a3b25e-0e3e-43bf-bd72-bea968a5ab99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:18:22.856048 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=100 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:18:22.886931 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 178/356] 10.222.0.49 () {58 vars in 1023 bytes} [Wed Jun 15 10:18:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 10:18:22.888749 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a3b25e-0e3e-43bf-bd72-bea968a5ab99 None None] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:18:22.916107 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced1292d-5761-4a0d-89ca-333cedd7d055 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:18:22.964868 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2c9cb0-85fe-481e-9131-d943bdecfd32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:18:22.966342 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2c9cb0-85fe-481e-9131-d943bdecfd32 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:22.966892 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2c9cb0-85fe-481e-9131-d943bdecfd32 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:22.967272 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2c9cb0-85fe-481e-9131-d943bdecfd32 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:22.982036 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd2c9cb0-85fe-481e-9131-d943bdecfd32 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 10:18:22.991571 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application [None req-fd2c9cb0-85fe-481e-9131-d943bdecfd32 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 15 10:18:22.991571 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:18:22.991571 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:18:22.991571 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:18:22.991571 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:18:22.991571 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:18:22.991571 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:18:22.991571 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:18:22.991571 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:18:22.991571 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:18:22.991571 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:18:22.991571 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:18:22.991571 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 10:18:22.991571 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 10:18:22.991571 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 10:18:22.991571 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 10:18:22.992536 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 10:18:22.992536 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 10:18:22.992536 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 10:18:22.992536 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 10:18:22.992536 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:18:22.992536 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:18:22.992536 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:18:22.992536 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:18:22.992536 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:18:22.992536 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:18:22.992536 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:18:22.992536 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application with Lock( Jun 15 10:18:22.992536 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:18:22.992536 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:18:22.992536 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:18:22.992536 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:18:22.993338 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:18:22.993338 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:18:22.993338 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:18:22.993338 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:18:22.993338 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 10:18:22.993338 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 10:18:22.993338 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 10:18:22.993338 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 10:18:22.993338 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 10:18:22.993338 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 10:18:22.993338 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 15 10:18:22.993338 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Jun 15 10:18:22.993338 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 179/357] 10.222.0.49 () {56 vars in 1099 bytes} [Wed Jun 15 10:18:22 2022] GET /identity/v3/roles/reader => generated 84 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:18:22.994004 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2c9cb0-85fe-481e-9131-d943bdecfd32 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:18:23.024832 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a3b25e-0e3e-43bf-bd72-bea968a5ab99 None None] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:18:23.044089 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:18:23.068090 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cdec8b8-021b-46e5-94de-956cf8bd43d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:18:23.068977 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdec8b8-021b-46e5-94de-956cf8bd43d0 None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:23.069264 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdec8b8-021b-46e5-94de-956cf8bd43d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:23.069543 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdec8b8-021b-46e5-94de-956cf8bd43d0 None admin] PATH_INFO: `/v3/groups/77182ac3056944e19934f133b3f7b1b3` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:23.070745 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cdec8b8-021b-46e5-94de-956cf8bd43d0 None admin] RBAC: Authorizing `identity:get_group(group_id=77182ac3056944e19934f133b3f7b1b3)` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:18:23.072386 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cdec8b8-021b-46e5-94de-956cf8bd43d0 None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:18:23.074038 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 179/358] 10.222.0.49 () {56 vars in 1180 bytes} [Wed Jun 15 10:18:22 2022] GET /identity/v3/groups/77182ac3056944e19934f133b3f7b1b3 => generated 226 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:18:23.074822 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cdec8b8-021b-46e5-94de-956cf8bd43d0 None admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:18:23.127006 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2c9cb0-85fe-481e-9131-d943bdecfd32 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:18:23.128547 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=100 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:18:23.172668 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abc2af1-7c8a-43b3-a5ac-dcc2c6d8ce49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:18:23.172668 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abc2af1-7c8a-43b3-a5ac-dcc2c6d8ce49 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:23.172668 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abc2af1-7c8a-43b3-a5ac-dcc2c6d8ce49 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:23.172668 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abc2af1-7c8a-43b3-a5ac-dcc2c6d8ce49 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:23.191598 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cdec8b8-021b-46e5-94de-956cf8bd43d0 None admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:18:23.238998 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2132724d-39c7-45dc-92ff-61059b740f74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:18:23.240314 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2132724d-39c7-45dc-92ff-61059b740f74 None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:23.240821 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2132724d-39c7-45dc-92ff-61059b740f74 None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:23.241306 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2132724d-39c7-45dc-92ff-61059b740f74 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:23.257163 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2132724d-39c7-45dc-92ff-61059b740f74 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 10:18:23.261822 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9abc2af1-7c8a-43b3-a5ac-dcc2c6d8ce49 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:18:23.264050 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9abc2af1-7c8a-43b3-a5ac-dcc2c6d8ce49 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:18:23.266883 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application [None req-2132724d-39c7-45dc-92ff-61059b740f74 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 15 10:18:23.266883 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:18:23.266883 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:18:23.266883 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:18:23.266883 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:18:23.266883 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:18:23.266883 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:18:23.266883 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:18:23.266883 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:18:23.266883 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:18:23.266883 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:18:23.266883 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:18:23.266883 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 10:18:23.266883 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 10:18:23.266883 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 10:18:23.266883 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 10:18:23.267872 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 10:18:23.267872 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 10:18:23.267872 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 10:18:23.267872 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 10:18:23.267872 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:18:23.267872 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:18:23.267872 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:18:23.267872 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:18:23.267872 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:18:23.267872 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:18:23.267872 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:18:23.267872 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application with Lock( Jun 15 10:18:23.267872 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:18:23.267872 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:18:23.267872 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:18:23.267872 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:18:23.268669 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:18:23.268669 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:18:23.268669 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:18:23.268669 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:18:23.268669 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 10:18:23.268669 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 10:18:23.268669 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 10:18:23.268669 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 10:18:23.268669 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 10:18:23.268669 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 10:18:23.268669 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 15 10:18:23.268669 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application Jun 15 10:18:23.273356 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 180/359] 10.222.0.49 () {56 vars in 1114 bytes} [Wed Jun 15 10:18:23 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:18:23.275732 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2132724d-39c7-45dc-92ff-61059b740f74 None admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:18:23.304275 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:18:23.405443 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 180/360] 10.222.0.49 () {58 vars in 1282 bytes} [Wed Jun 15 10:18:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 412 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 15 10:18:23.407256 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abc2af1-7c8a-43b3-a5ac-dcc2c6d8ce49 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:18:23.412533 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2132724d-39c7-45dc-92ff-61059b740f74 None admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:18:23.425354 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=100 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:18:23.461601 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07797fc-b140-464a-93aa-ca6d51716d50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:18:23.462280 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07797fc-b140-464a-93aa-ca6d51716d50 None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:23.462893 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07797fc-b140-464a-93aa-ca6d51716d50 None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:23.466199 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07797fc-b140-464a-93aa-ca6d51716d50 None admin] PATH_INFO: `/v3/roles` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:23.466199 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07797fc-b140-464a-93aa-ca6d51716d50 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:18:23.466199 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07797fc-b140-464a-93aa-ca6d51716d50 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:18:23.466588 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07797fc-b140-464a-93aa-ca6d51716d50 None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:18:23.476697 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 181/361] 10.222.0.49 () {56 vars in 1101 bytes} [Wed Jun 15 10:18:23 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:18:23.477562 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07797fc-b140-464a-93aa-ca6d51716d50 None admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:18:23.536914 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abc2af1-7c8a-43b3-a5ac-dcc2c6d8ce49 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:18:23.575877 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:18:23.582188 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88fd39f-6af9-4454-b0cc-fe5abc87a22c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:18:23.582860 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88fd39f-6af9-4454-b0cc-fe5abc87a22c None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:23.583199 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88fd39f-6af9-4454-b0cc-fe5abc87a22c None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:23.583452 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88fd39f-6af9-4454-b0cc-fe5abc87a22c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:23.587035 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88fd39f-6af9-4454-b0cc-fe5abc87a22c None admin] RBAC: Adding query filter params (group.id=77182ac3056944e19934f133b3f7b1b3, role.id=6876e57cc51c448097ab109d35b2a3da, scope.project.id=df1f045b58c24b4dba0964eff7d2e540) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:18:23.587035 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88fd39f-6af9-4454-b0cc-fe5abc87a22c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:18:23.589282 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88fd39f-6af9-4454-b0cc-fe5abc87a22c None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:18:23.594561 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 181/362] 10.222.0.49 () {56 vars in 1376 bytes} [Wed Jun 15 10:18:23 2022] GET /identity/v3/role_assignments?group.id=77182ac3056944e19934f133b3f7b1b3&scope.project.id=df1f045b58c24b4dba0964eff7d2e540&role.id=6876e57cc51c448097ab109d35b2a3da => generated 260 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:18:23.596303 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f349f7-e53a-42fc-829f-4a0a9d8c9e52 None None] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:23.596828 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f349f7-e53a-42fc-829f-4a0a9d8c9e52 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:23.597452 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f349f7-e53a-42fc-829f-4a0a9d8c9e52 None None] PATH_INFO: `/` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:23.599251 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 182/363] 10.222.0.49 () {54 vars in 926 bytes} [Wed Jun 15 10:18:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:18:23.600298 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f349f7-e53a-42fc-829f-4a0a9d8c9e52 None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:18:23.614039 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07797fc-b140-464a-93aa-ca6d51716d50 None admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:18:23.654965 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f5b126-79e1-4611-814f-669285fbde2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:18:23.655705 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f5b126-79e1-4611-814f-669285fbde2a None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:23.656000 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f5b126-79e1-4611-814f-669285fbde2a None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:23.656289 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f5b126-79e1-4611-814f-669285fbde2a None admin] PATH_INFO: `/v3/roles` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:23.657071 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f5b126-79e1-4611-814f-669285fbde2a None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:18:23.657391 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f5b126-79e1-4611-814f-669285fbde2a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:18:23.658846 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f5b126-79e1-4611-814f-669285fbde2a None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:18:23.674896 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 182/364] 10.222.0.49 () {56 vars in 1111 bytes} [Wed Jun 15 10:18:23 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:18:23.674896 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f5b126-79e1-4611-814f-669285fbde2a None admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:18:23.707553 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=100 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:18:23.727810 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f349f7-e53a-42fc-829f-4a0a9d8c9e52 None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:18:23.767004 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb611c1-3182-405f-aa7b-8e988e5b4849 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:18:23.768699 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb611c1-3182-405f-aa7b-8e988e5b4849 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:23.769167 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb611c1-3182-405f-aa7b-8e988e5b4849 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:23.770152 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb611c1-3182-405f-aa7b-8e988e5b4849 None admin] PATH_INFO: `/v3/projects/5f3cd857995e418386bb22cdf90c1ae9` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:23.771703 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb611c1-3182-405f-aa7b-8e988e5b4849 None admin] RBAC: Authorizing `identity:get_project(project_id=5f3cd857995e418386bb22cdf90c1ae9)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:18:23.773389 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb611c1-3182-405f-aa7b-8e988e5b4849 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:18:23.780456 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 183/365] 10.222.0.49 () {56 vars in 1186 bytes} [Wed Jun 15 10:18:23 2022] GET /identity/v3/projects/5f3cd857995e418386bb22cdf90c1ae9 => generated 298 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:18:23.782196 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a6c260-3d11-4d6a-8fe2-c76e82de0b79 None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:23.782694 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a6c260-3d11-4d6a-8fe2-c76e82de0b79 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:23.783196 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a6c260-3d11-4d6a-8fe2-c76e82de0b79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:23.792941 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f5b126-79e1-4611-814f-669285fbde2a None admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:18:23.805061 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-a6a6c260-3d11-4d6a-8fe2-c76e82de0b79 None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:18:23.806589 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-a6a6c260-3d11-4d6a-8fe2-c76e82de0b79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:18:23.835420 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f971672-696f-4fbd-a7a1-40756fbdb64b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:18:23.836368 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f971672-696f-4fbd-a7a1-40756fbdb64b None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:23.836657 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f971672-696f-4fbd-a7a1-40756fbdb64b None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:23.836938 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f971672-696f-4fbd-a7a1-40756fbdb64b None admin] PATH_INFO: `/v3/users/948979771ffa47d184a627149510a973` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:23.838949 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f971672-696f-4fbd-a7a1-40756fbdb64b None admin] RBAC: Authorizing `identity:get_user(user_id=948979771ffa47d184a627149510a973)` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:18:23.839688 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f971672-696f-4fbd-a7a1-40756fbdb64b None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:18:23.846898 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 183/366] 10.222.0.49 () {56 vars in 1177 bytes} [Wed Jun 15 10:18:23 2022] GET /identity/v3/users/948979771ffa47d184a627149510a973 => generated 293 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:18:23.846898 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f971672-696f-4fbd-a7a1-40756fbdb64b None admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:18:23.885674 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:18:23.898553 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 184/367] 10.222.0.49 () {58 vars in 1023 bytes} [Wed Jun 15 10:18:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 10:18:23.899487 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a6c260-3d11-4d6a-8fe2-c76e82de0b79 None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:18:23.991314 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=100 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:18:23.999480 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f971672-696f-4fbd-a7a1-40756fbdb64b None admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:18:24.034884 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a6c260-3d11-4d6a-8fe2-c76e82de0b79 None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:18:24.040127 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a19e6f-11bf-40d9-a9a4-f93110af3986 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:18:24.041086 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a19e6f-11bf-40d9-a9a4-f93110af3986 None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:24.041572 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a19e6f-11bf-40d9-a9a4-f93110af3986 None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:24.042063 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a19e6f-11bf-40d9-a9a4-f93110af3986 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:24.076482 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2017a0-9adc-4cc8-9061-12fea24ad165 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:18:24.077277 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2017a0-9adc-4cc8-9061-12fea24ad165 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:24.077557 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2017a0-9adc-4cc8-9061-12fea24ad165 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:24.077832 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2017a0-9adc-4cc8-9061-12fea24ad165 None admin] PATH_INFO: `/v3/groups/77182ac3056944e19934f133b3f7b1b3` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:24.079028 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb2017a0-9adc-4cc8-9061-12fea24ad165 None admin] RBAC: Authorizing `identity:get_group(group_id=77182ac3056944e19934f133b3f7b1b3)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:18:24.080502 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb2017a0-9adc-4cc8-9061-12fea24ad165 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:18:24.082421 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 185/368] 10.222.0.49 () {56 vars in 1180 bytes} [Wed Jun 15 10:18:23 2022] GET /identity/v3/groups/77182ac3056944e19934f133b3f7b1b3 => generated 226 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:18:24.087935 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2017a0-9adc-4cc8-9061-12fea24ad165 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:18:24.134650 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a19e6f-11bf-40d9-a9a4-f93110af3986 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:18:24.137201 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a19e6f-11bf-40d9-a9a4-f93110af3986 None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:18:24.156747 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:18:24.208175 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2017a0-9adc-4cc8-9061-12fea24ad165 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:18:24.256457 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d26338-62cd-44cc-82ac-63c4d5c3bd10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:18:24.256792 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d26338-62cd-44cc-82ac-63c4d5c3bd10 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:24.257081 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d26338-62cd-44cc-82ac-63c4d5c3bd10 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:24.257356 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d26338-62cd-44cc-82ac-63c4d5c3bd10 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:24.258257 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d26338-62cd-44cc-82ac-63c4d5c3bd10 None admin] RBAC: Adding query filter params (role.id=301a3c8a23ba40edbc35df1da8af0de9, user.id=948979771ffa47d184a627149510a973, scope.system=all) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:18:24.258570 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d26338-62cd-44cc-82ac-63c4d5c3bd10 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:18:24.260117 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d26338-62cd-44cc-82ac-63c4d5c3bd10 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:18:24.272294 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=100 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:18:24.283117 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 186/369] 10.222.0.49 () {56 vars in 1308 bytes} [Wed Jun 15 10:18:24 2022] GET /identity/v3/role_assignments?user.id=948979771ffa47d184a627149510a973&scope.system=all&role.id=301a3c8a23ba40edbc35df1da8af0de9 => generated 510 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:18:24.283117 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d26338-62cd-44cc-82ac-63c4d5c3bd10 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:18:24.285839 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 184/370] 10.222.0.49 () {58 vars in 1282 bytes} [Wed Jun 15 10:18:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 443 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 15 10:18:24.286997 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a19e6f-11bf-40d9-a9a4-f93110af3986 None admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:18:24.404783 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d26338-62cd-44cc-82ac-63c4d5c3bd10 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:18:24.430832 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:18:24.456773 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d83984f-5f0e-45e2-a82b-af0b397fbafa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:18:24.457599 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d83984f-5f0e-45e2-a82b-af0b397fbafa None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:24.457885 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d83984f-5f0e-45e2-a82b-af0b397fbafa None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:24.458164 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d83984f-5f0e-45e2-a82b-af0b397fbafa None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:24.489316 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d83984f-5f0e-45e2-a82b-af0b397fbafa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 10:18:24.495792 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a19e6f-11bf-40d9-a9a4-f93110af3986 None admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:18:24.498334 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application [None req-6d83984f-5f0e-45e2-a82b-af0b397fbafa None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 10:18:24.498334 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:18:24.498334 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:18:24.498334 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:18:24.498334 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:18:24.498334 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:18:24.498334 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:18:24.498334 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:18:24.498334 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:18:24.498334 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:18:24.498334 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:18:24.498334 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:18:24.498334 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 10:18:24.498334 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 10:18:24.498334 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 10:18:24.498334 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 10:18:24.499250 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 10:18:24.499250 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 10:18:24.499250 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 10:18:24.499250 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 10:18:24.499250 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:18:24.499250 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:18:24.499250 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:18:24.499250 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:18:24.499250 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:18:24.499250 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:18:24.499250 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:18:24.499250 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application with Lock( Jun 15 10:18:24.499250 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:18:24.499250 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:18:24.499250 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:18:24.499250 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:18:24.500045 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:18:24.500045 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:18:24.500045 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:18:24.500045 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:18:24.500045 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 10:18:24.500045 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 10:18:24.500045 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 10:18:24.500045 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 10:18:24.500045 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 10:18:24.500045 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 10:18:24.500045 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 10:18:24.500045 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Jun 15 10:18:24.502719 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 187/371] 10.222.0.49 () {56 vars in 1096 bytes} [Wed Jun 15 10:18:24 2022] GET /identity/v3/roles/admin => generated 83 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:18:24.503906 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d83984f-5f0e-45e2-a82b-af0b397fbafa None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:18:24.541807 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34073fe-89e4-4a04-ab8d-11d66eb99698 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:18:24.542526 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34073fe-89e4-4a04-ab8d-11d66eb99698 None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:24.542822 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34073fe-89e4-4a04-ab8d-11d66eb99698 None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:24.543113 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34073fe-89e4-4a04-ab8d-11d66eb99698 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:24.547118 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d34073fe-89e4-4a04-ab8d-11d66eb99698 None admin] RBAC: Adding query filter params (group.id=77182ac3056944e19934f133b3f7b1b3, role.id=4d2118569d074a288b37d120110c300d, scope.project.id=5f3cd857995e418386bb22cdf90c1ae9) {{(pid=80119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:18:24.547118 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d34073fe-89e4-4a04-ab8d-11d66eb99698 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:18:24.547118 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d34073fe-89e4-4a04-ab8d-11d66eb99698 None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:18:24.553478 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 185/372] 10.222.0.49 () {56 vars in 1376 bytes} [Wed Jun 15 10:18:24 2022] GET /identity/v3/role_assignments?group.id=77182ac3056944e19934f133b3f7b1b3&scope.project.id=5f3cd857995e418386bb22cdf90c1ae9&role.id=4d2118569d074a288b37d120110c300d => generated 260 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:18:24.554588 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34073fe-89e4-4a04-ab8d-11d66eb99698 None admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:18:24.630211 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=100 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:18:24.677739 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34073fe-89e4-4a04-ab8d-11d66eb99698 None admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:18:24.689645 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d83984f-5f0e-45e2-a82b-af0b397fbafa None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:18:24.719595 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964f9c3a-d337-46e7-9022-dad19e954e09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:18:24.720535 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964f9c3a-d337-46e7-9022-dad19e954e09 None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:24.720822 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964f9c3a-d337-46e7-9022-dad19e954e09 None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:24.721100 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964f9c3a-d337-46e7-9022-dad19e954e09 None admin] PATH_INFO: `/v3/roles` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:24.723302 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-964f9c3a-d337-46e7-9022-dad19e954e09 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:18:24.723617 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-964f9c3a-d337-46e7-9022-dad19e954e09 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:18:24.725034 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-964f9c3a-d337-46e7-9022-dad19e954e09 None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:18:24.735056 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0912995c-32b7-4e40-9b70-459988ac25d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:18:24.735469 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 186/373] 10.222.0.49 () {56 vars in 1099 bytes} [Wed Jun 15 10:18:24 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:18:24.736632 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0912995c-32b7-4e40-9b70-459988ac25d8 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:24.737118 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0912995c-32b7-4e40-9b70-459988ac25d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:24.737588 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0912995c-32b7-4e40-9b70-459988ac25d8 None admin] PATH_INFO: `/v3/projects/df1f045b58c24b4dba0964eff7d2e540` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:24.739129 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0912995c-32b7-4e40-9b70-459988ac25d8 None admin] RBAC: Authorizing `identity:get_project(project_id=df1f045b58c24b4dba0964eff7d2e540)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:18:24.740712 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964f9c3a-d337-46e7-9022-dad19e954e09 None admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:18:24.741498 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0912995c-32b7-4e40-9b70-459988ac25d8 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:18:24.743679 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 188/374] 10.222.0.49 () {56 vars in 1186 bytes} [Wed Jun 15 10:18:24 2022] GET /identity/v3/projects/df1f045b58c24b4dba0964eff7d2e540 => generated 302 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:18:24.749587 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0912995c-32b7-4e40-9b70-459988ac25d8 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:18:24.832370 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:18:24.902459 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0912995c-32b7-4e40-9b70-459988ac25d8 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:18:24.906184 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964f9c3a-d337-46e7-9022-dad19e954e09 None admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:18:24.944201 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=100 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:18:24.951594 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9924475-1953-47c1-82e4-9c009428e585 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:18:24.952575 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9924475-1953-47c1-82e4-9c009428e585 None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:24.953062 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9924475-1953-47c1-82e4-9c009428e585 None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:24.953555 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9924475-1953-47c1-82e4-9c009428e585 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:24.972669 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733e7133-45c1-4a45-aabc-f609b3e5e074 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:18:24.973550 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733e7133-45c1-4a45-aabc-f609b3e5e074 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:24.973839 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733e7133-45c1-4a45-aabc-f609b3e5e074 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:24.974121 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733e7133-45c1-4a45-aabc-f609b3e5e074 None admin] PATH_INFO: `/v3/groups/77182ac3056944e19934f133b3f7b1b3` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:24.979820 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733e7133-45c1-4a45-aabc-f609b3e5e074 None admin] RBAC: Authorizing `identity:get_group(group_id=77182ac3056944e19934f133b3f7b1b3)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:18:24.981335 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733e7133-45c1-4a45-aabc-f609b3e5e074 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:18:24.983569 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 189/375] 10.222.0.49 () {56 vars in 1180 bytes} [Wed Jun 15 10:18:24 2022] GET /identity/v3/groups/77182ac3056944e19934f133b3f7b1b3 => generated 226 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 10:18:24.988922 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733e7133-45c1-4a45-aabc-f609b3e5e074 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:18:25.060356 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9924475-1953-47c1-82e4-9c009428e585 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:18:25.063640 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9924475-1953-47c1-82e4-9c009428e585 None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:18:25.190001 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733e7133-45c1-4a45-aabc-f609b3e5e074 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:18:25.225425 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:18:25.259081 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc2d7a6-a4b5-40ff-b818-ec9006005caf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:18:25.259791 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc2d7a6-a4b5-40ff-b818-ec9006005caf None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:25.260247 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc2d7a6-a4b5-40ff-b818-ec9006005caf None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:25.260460 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 187/376] 10.222.0.49 () {58 vars in 1282 bytes} [Wed Jun 15 10:18:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 520 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 15 10:18:25.260961 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc2d7a6-a4b5-40ff-b818-ec9006005caf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:25.262067 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc2d7a6-a4b5-40ff-b818-ec9006005caf None admin] RBAC: Adding query filter params (group.id=77182ac3056944e19934f133b3f7b1b3, role.id=4d2118569d074a288b37d120110c300d, scope.project.id=df1f045b58c24b4dba0964eff7d2e540) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:18:25.263175 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc2d7a6-a4b5-40ff-b818-ec9006005caf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:18:25.264884 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc2d7a6-a4b5-40ff-b818-ec9006005caf None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:18:25.266459 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9924475-1953-47c1-82e4-9c009428e585 None admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:18:25.272399 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 190/377] 10.222.0.49 () {56 vars in 1376 bytes} [Wed Jun 15 10:18:24 2022] GET /identity/v3/role_assignments?group.id=77182ac3056944e19934f133b3f7b1b3&scope.project.id=df1f045b58c24b4dba0964eff7d2e540&role.id=4d2118569d074a288b37d120110c300d => generated 260 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 10:18:25.305538 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=100 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:18:25.346995 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3985f9f-b463-4f6e-87e8-ed2c32079459 None None] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:25.347664 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3985f9f-b463-4f6e-87e8-ed2c32079459 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:25.348151 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3985f9f-b463-4f6e-87e8-ed2c32079459 None None] PATH_INFO: `/` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:25.349776 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 191/378] 10.222.0.49 () {54 vars in 926 bytes} [Wed Jun 15 10:18:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 15 10:18:25.360666 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b12bb5-1625-4008-86a1-4732d4fb155c None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:25.361304 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b12bb5-1625-4008-86a1-4732d4fb155c None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:25.361788 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b12bb5-1625-4008-86a1-4732d4fb155c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:25.396357 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-27b12bb5-1625-4008-86a1-4732d4fb155c None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:18:25.397917 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-27b12bb5-1625-4008-86a1-4732d4fb155c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:18:25.407118 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9924475-1953-47c1-82e4-9c009428e585 None admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:18:25.455300 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237f7943-fe32-406c-bd96-0df1e9bf69e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:18:25.456337 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237f7943-fe32-406c-bd96-0df1e9bf69e4 None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:25.456625 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237f7943-fe32-406c-bd96-0df1e9bf69e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:25.456899 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237f7943-fe32-406c-bd96-0df1e9bf69e4 None admin] PATH_INFO: `/v3/projects/c8da8e1de64f469c9cc5f9a5f416060d` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:25.458151 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237f7943-fe32-406c-bd96-0df1e9bf69e4 None admin] RBAC: Authorizing `identity:get_project(project_id=c8da8e1de64f469c9cc5f9a5f416060d)` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:18:25.459663 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237f7943-fe32-406c-bd96-0df1e9bf69e4 None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:18:25.461385 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 188/379] 10.222.0.49 () {56 vars in 1186 bytes} [Wed Jun 15 10:18:25 2022] GET /identity/v3/projects/c8da8e1de64f469c9cc5f9a5f416060d => generated 344 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 10:18:25.467276 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237f7943-fe32-406c-bd96-0df1e9bf69e4 None admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:18:25.499399 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 192/380] 10.222.0.49 () {58 vars in 1023 bytes} [Wed Jun 15 10:18:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 15 10:18:25.539154 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b12bb5-1625-4008-86a1-4732d4fb155c None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:18:25.600549 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=100 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:18:25.610948 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237f7943-fe32-406c-bd96-0df1e9bf69e4 None admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:18:25.650586 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d83d4a8-3160-4a7d-854b-f4915d392e01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:18:25.651717 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d83d4a8-3160-4a7d-854b-f4915d392e01 None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:25.652170 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d83d4a8-3160-4a7d-854b-f4915d392e01 None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:25.652758 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d83d4a8-3160-4a7d-854b-f4915d392e01 None admin] PATH_INFO: `/v3/groups/ffc16ae2b8b843bc92af79734c11eeba` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:25.658759 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b12bb5-1625-4008-86a1-4732d4fb155c None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:18:25.659458 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d83d4a8-3160-4a7d-854b-f4915d392e01 None admin] RBAC: Authorizing `identity:get_group(group_id=ffc16ae2b8b843bc92af79734c11eeba)` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:18:25.661094 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d83d4a8-3160-4a7d-854b-f4915d392e01 None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:18:25.663353 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 189/381] 10.222.0.49 () {56 vars in 1180 bytes} [Wed Jun 15 10:18:25 2022] GET /identity/v3/groups/ffc16ae2b8b843bc92af79734c11eeba => generated 229 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:18:25.669945 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d83d4a8-3160-4a7d-854b-f4915d392e01 None admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:18:25.672507 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:18:25.698927 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f397cd2-5243-4e2a-a87e-b30e3bd63a06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:18:25.700159 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f397cd2-5243-4e2a-a87e-b30e3bd63a06 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:25.700655 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f397cd2-5243-4e2a-a87e-b30e3bd63a06 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:25.701133 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f397cd2-5243-4e2a-a87e-b30e3bd63a06 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:25.712568 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f397cd2-5243-4e2a-a87e-b30e3bd63a06 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 10:18:25.720841 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application [None req-1f397cd2-5243-4e2a-a87e-b30e3bd63a06 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 15 10:18:25.720841 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:18:25.720841 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:18:25.720841 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:18:25.720841 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:18:25.720841 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:18:25.720841 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:18:25.720841 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:18:25.720841 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:18:25.720841 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:18:25.720841 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:18:25.720841 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:18:25.720841 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 10:18:25.720841 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 10:18:25.720841 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 10:18:25.720841 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 10:18:25.721752 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 10:18:25.721752 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 10:18:25.721752 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 10:18:25.721752 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 10:18:25.721752 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:18:25.721752 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:18:25.721752 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:18:25.721752 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:18:25.721752 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:18:25.721752 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:18:25.721752 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:18:25.721752 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application with Lock( Jun 15 10:18:25.721752 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:18:25.721752 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:18:25.721752 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:18:25.721752 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:18:25.722552 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:18:25.722552 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:18:25.722552 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:18:25.722552 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:18:25.722552 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 10:18:25.722552 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 10:18:25.722552 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 10:18:25.722552 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 10:18:25.722552 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 10:18:25.722552 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 10:18:25.722552 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 15 10:18:25.722552 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Jun 15 10:18:25.725282 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 193/382] 10.222.0.49 () {56 vars in 1099 bytes} [Wed Jun 15 10:18:25 2022] GET /identity/v3/roles/member => generated 84 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 10:18:25.730943 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f397cd2-5243-4e2a-a87e-b30e3bd63a06 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:18:25.790499 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d83d4a8-3160-4a7d-854b-f4915d392e01 None admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:18:25.833454 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29062f4b-c8bc-4d8e-891a-ecfd00dfb3c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:18:25.834017 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29062f4b-c8bc-4d8e-891a-ecfd00dfb3c8 None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:25.834308 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29062f4b-c8bc-4d8e-891a-ecfd00dfb3c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:25.834592 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29062f4b-c8bc-4d8e-891a-ecfd00dfb3c8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:25.835470 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29062f4b-c8bc-4d8e-891a-ecfd00dfb3c8 None admin] RBAC: Adding query filter params (group.id=ffc16ae2b8b843bc92af79734c11eeba, role.id=8051174e7a7947d5b991ebd793c2e543, scope.project.id=c8da8e1de64f469c9cc5f9a5f416060d) {{(pid=80119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:18:25.835790 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29062f4b-c8bc-4d8e-891a-ecfd00dfb3c8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:18:25.837243 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29062f4b-c8bc-4d8e-891a-ecfd00dfb3c8 None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:18:25.844772 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f397cd2-5243-4e2a-a87e-b30e3bd63a06 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:18:25.851565 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 190/383] 10.222.0.49 () {56 vars in 1376 bytes} [Wed Jun 15 10:18:25 2022] GET /identity/v3/role_assignments?group.id=ffc16ae2b8b843bc92af79734c11eeba&scope.project.id=c8da8e1de64f469c9cc5f9a5f416060d&role.id=8051174e7a7947d5b991ebd793c2e543 => generated 260 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 10:18:25.886084 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289d7163-12b8-4a9e-bd2b-dfbc5465bfdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:18:25.887169 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289d7163-12b8-4a9e-bd2b-dfbc5465bfdf None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:25.888225 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289d7163-12b8-4a9e-bd2b-dfbc5465bfdf None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:25.888728 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289d7163-12b8-4a9e-bd2b-dfbc5465bfdf None admin] PATH_INFO: `/v3/roles` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:25.889704 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289d7163-12b8-4a9e-bd2b-dfbc5465bfdf None admin] RBAC: Adding query filter params (name=member) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:18:25.890221 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289d7163-12b8-4a9e-bd2b-dfbc5465bfdf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:18:25.891953 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289d7163-12b8-4a9e-bd2b-dfbc5465bfdf None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:18:25.902409 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 194/384] 10.222.0.49 () {56 vars in 1101 bytes} [Wed Jun 15 10:18:25 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:18:25.909359 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29062f4b-c8bc-4d8e-891a-ecfd00dfb3c8 None admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:18:25.938822 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=100 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:18:26.051899 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29062f4b-c8bc-4d8e-891a-ecfd00dfb3c8 None admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:18:26.088349 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f0f485-8324-4ef5-90e6-139228436062 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:18:26.089220 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f0f485-8324-4ef5-90e6-139228436062 None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:26.089507 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f0f485-8324-4ef5-90e6-139228436062 None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:26.089874 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f0f485-8324-4ef5-90e6-139228436062 None admin] PATH_INFO: `/v3/groups/77182ac3056944e19934f133b3f7b1b3` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:26.091147 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f0f485-8324-4ef5-90e6-139228436062 None admin] RBAC: Authorizing `identity:get_group(group_id=77182ac3056944e19934f133b3f7b1b3)` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:18:26.092738 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f0f485-8324-4ef5-90e6-139228436062 None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:18:26.099779 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 191/385] 10.222.0.49 () {56 vars in 1180 bytes} [Wed Jun 15 10:18:25 2022] GET /identity/v3/groups/77182ac3056944e19934f133b3f7b1b3 => generated 226 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:18:26.105386 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289d7163-12b8-4a9e-bd2b-dfbc5465bfdf None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:18:26.156129 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:18:26.236457 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289d7163-12b8-4a9e-bd2b-dfbc5465bfdf None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:18:26.281199 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5215dfad-cb2d-44fa-8333-f08678424876 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:18:26.282431 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5215dfad-cb2d-44fa-8333-f08678424876 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:26.282921 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5215dfad-cb2d-44fa-8333-f08678424876 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:26.283472 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5215dfad-cb2d-44fa-8333-f08678424876 None admin] PATH_INFO: `/v3/projects/5f3cd857995e418386bb22cdf90c1ae9` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:26.284925 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5215dfad-cb2d-44fa-8333-f08678424876 None admin] RBAC: Authorizing `identity:get_project(project_id=5f3cd857995e418386bb22cdf90c1ae9)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:18:26.286596 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5215dfad-cb2d-44fa-8333-f08678424876 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:18:26.288913 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 195/386] 10.222.0.49 () {56 vars in 1186 bytes} [Wed Jun 15 10:18:26 2022] GET /identity/v3/projects/5f3cd857995e418386bb22cdf90c1ae9 => generated 298 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:18:26.294405 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f0f485-8324-4ef5-90e6-139228436062 None admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:18:26.428078 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=100 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:18:26.442472 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f0f485-8324-4ef5-90e6-139228436062 None admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:18:26.484473 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87aadd32-bec5-4bd1-b69f-ecefde30a9ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:18:26.485404 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87aadd32-bec5-4bd1-b69f-ecefde30a9ae None admin] REQUEST_METHOD: `PUT` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:26.485695 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87aadd32-bec5-4bd1-b69f-ecefde30a9ae None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:26.486012 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87aadd32-bec5-4bd1-b69f-ecefde30a9ae None admin] PATH_INFO: `/v3/projects/5f3cd857995e418386bb22cdf90c1ae9/groups/77182ac3056944e19934f133b3f7b1b3/roles/6876e57cc51c448097ab109d35b2a3da` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:26.488627 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87aadd32-bec5-4bd1-b69f-ecefde30a9ae None admin] RBAC: Authorizing `identity:create_grant(project_id=5f3cd857995e418386bb22cdf90c1ae9, group_id=77182ac3056944e19934f133b3f7b1b3, role_id=6876e57cc51c448097ab109d35b2a3da)` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:18:26.490174 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87aadd32-bec5-4bd1-b69f-ecefde30a9ae None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:18:26.515158 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 192/387] 10.222.0.49 () {58 vars in 1442 bytes} [Wed Jun 15 10:18:26 2022] PUT /identity/v3/projects/5f3cd857995e418386bb22cdf90c1ae9/groups/77182ac3056944e19934f133b3f7b1b3/roles/6876e57cc51c448097ab109d35b2a3da => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:18:26.569310 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f595b30-74d4-4111-b5ad-cd56a29a5425 None None] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:26.570071 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f595b30-74d4-4111-b5ad-cd56a29a5425 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:26.570558 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f595b30-74d4-4111-b5ad-cd56a29a5425 None None] PATH_INFO: `/` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:26.572258 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 196/388] 10.222.0.49 () {54 vars in 926 bytes} [Wed Jun 15 10:18:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:18:26.578063 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf055de-7f99-4bc3-8092-d213ab598114 None None] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:26.578356 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf055de-7f99-4bc3-8092-d213ab598114 None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:26.578643 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf055de-7f99-4bc3-8092-d213ab598114 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:26.603441 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.auth.core [None req-acf055de-7f99-4bc3-8092-d213ab598114 None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:18:26.604769 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-acf055de-7f99-4bc3-8092-d213ab598114 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:18:26.701027 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 193/389] 10.222.0.49 () {58 vars in 1023 bytes} [Wed Jun 15 10:18:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 10:18:26.737637 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f595b30-74d4-4111-b5ad-cd56a29a5425 None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:18:26.870437 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:18:26.871772 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f595b30-74d4-4111-b5ad-cd56a29a5425 None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:18:26.916269 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e4a6bb-51f4-4c15-b2fa-00e1bf2311e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:18:26.917084 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e4a6bb-51f4-4c15-b2fa-00e1bf2311e1 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:26.917370 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e4a6bb-51f4-4c15-b2fa-00e1bf2311e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:26.917647 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e4a6bb-51f4-4c15-b2fa-00e1bf2311e1 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:26.930754 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e4a6bb-51f4-4c15-b2fa-00e1bf2311e1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 10:18:26.938828 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application [None req-e0e4a6bb-51f4-4c15-b2fa-00e1bf2311e1 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 15 10:18:26.938828 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:18:26.938828 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:18:26.938828 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:18:26.938828 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:18:26.938828 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:18:26.938828 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:18:26.938828 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:18:26.938828 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:18:26.938828 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:18:26.938828 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:18:26.938828 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:18:26.938828 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 10:18:26.938828 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 10:18:26.938828 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 10:18:26.938828 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 10:18:26.939743 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 10:18:26.939743 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 10:18:26.939743 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 10:18:26.939743 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 10:18:26.939743 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:18:26.939743 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:18:26.939743 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:18:26.939743 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:18:26.939743 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:18:26.939743 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:18:26.939743 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:18:26.939743 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application with Lock( Jun 15 10:18:26.939743 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:18:26.939743 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:18:26.939743 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:18:26.939743 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:18:26.940526 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:18:26.940526 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:18:26.940526 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:18:26.940526 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:18:26.940526 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 10:18:26.940526 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 10:18:26.940526 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 10:18:26.940526 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 10:18:26.940526 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 10:18:26.940526 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 10:18:26.940526 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 15 10:18:26.940526 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Jun 15 10:18:26.941499 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 197/390] 10.222.0.49 () {56 vars in 1099 bytes} [Wed Jun 15 10:18:26 2022] GET /identity/v3/roles/member => generated 84 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:18:26.947407 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf055de-7f99-4bc3-8092-d213ab598114 None None] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:18:27.054853 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:18:27.083159 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf055de-7f99-4bc3-8092-d213ab598114 None None] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:18:27.132696 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6693917-f47a-4d10-a0d4-a8a460ec2680 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:18:27.133390 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6693917-f47a-4d10-a0d4-a8a460ec2680 None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:27.133683 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6693917-f47a-4d10-a0d4-a8a460ec2680 None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:27.133965 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6693917-f47a-4d10-a0d4-a8a460ec2680 None admin] PATH_INFO: `/v3/roles` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:27.134730 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6693917-f47a-4d10-a0d4-a8a460ec2680 None admin] RBAC: Adding query filter params (name=member) {{(pid=80119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:18:27.135040 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6693917-f47a-4d10-a0d4-a8a460ec2680 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:18:27.136465 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6693917-f47a-4d10-a0d4-a8a460ec2680 None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:18:27.150822 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 194/391] 10.222.0.49 () {56 vars in 1101 bytes} [Wed Jun 15 10:18:26 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:18:27.156635 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e4a6bb-51f4-4c15-b2fa-00e1bf2311e1 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:18:27.295557 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e4a6bb-51f4-4c15-b2fa-00e1bf2311e1 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:18:27.339984 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898c8352-1e6b-4ffe-9ed1-a1cc325156f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:18:27.341348 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898c8352-1e6b-4ffe-9ed1-a1cc325156f6 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:27.341635 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898c8352-1e6b-4ffe-9ed1-a1cc325156f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:27.341915 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898c8352-1e6b-4ffe-9ed1-a1cc325156f6 None admin] PATH_INFO: `/v3/groups/77182ac3056944e19934f133b3f7b1b3` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:27.343149 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-898c8352-1e6b-4ffe-9ed1-a1cc325156f6 None admin] RBAC: Authorizing `identity:get_group(group_id=77182ac3056944e19934f133b3f7b1b3)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:18:27.344607 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-898c8352-1e6b-4ffe-9ed1-a1cc325156f6 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:18:27.346569 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 198/392] 10.222.0.49 () {56 vars in 1180 bytes} [Wed Jun 15 10:18:27 2022] GET /identity/v3/groups/77182ac3056944e19934f133b3f7b1b3 => generated 226 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:18:27.351805 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6693917-f47a-4d10-a0d4-a8a460ec2680 None admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:18:27.540190 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6693917-f47a-4d10-a0d4-a8a460ec2680 None admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:18:27.584997 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e028312e-e238-4d0c-b405-b9e58a3b322d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:18:27.585868 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e028312e-e238-4d0c-b405-b9e58a3b322d None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:27.586148 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e028312e-e238-4d0c-b405-b9e58a3b322d None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:27.586420 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e028312e-e238-4d0c-b405-b9e58a3b322d None admin] PATH_INFO: `/v3/projects/df1f045b58c24b4dba0964eff7d2e540` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:27.591867 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e028312e-e238-4d0c-b405-b9e58a3b322d None admin] RBAC: Authorizing `identity:get_project(project_id=df1f045b58c24b4dba0964eff7d2e540)` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:18:27.593278 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e028312e-e238-4d0c-b405-b9e58a3b322d None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:18:27.595333 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 195/393] 10.222.0.49 () {56 vars in 1186 bytes} [Wed Jun 15 10:18:27 2022] GET /identity/v3/projects/df1f045b58c24b4dba0964eff7d2e540 => generated 302 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:18:27.600732 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898c8352-1e6b-4ffe-9ed1-a1cc325156f6 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:18:27.643773 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:18:27.736163 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898c8352-1e6b-4ffe-9ed1-a1cc325156f6 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:18:27.778773 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d12354-196f-4aaf-b3b0-bbd29e97f909 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:18:27.779699 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d12354-196f-4aaf-b3b0-bbd29e97f909 None admin] REQUEST_METHOD: `PUT` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:27.779984 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d12354-196f-4aaf-b3b0-bbd29e97f909 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:27.780261 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d12354-196f-4aaf-b3b0-bbd29e97f909 None admin] PATH_INFO: `/v3/projects/df1f045b58c24b4dba0964eff7d2e540/groups/77182ac3056944e19934f133b3f7b1b3/roles/6876e57cc51c448097ab109d35b2a3da` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:27.787876 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d12354-196f-4aaf-b3b0-bbd29e97f909 None admin] RBAC: Authorizing `identity:create_grant(project_id=df1f045b58c24b4dba0964eff7d2e540, group_id=77182ac3056944e19934f133b3f7b1b3, role_id=6876e57cc51c448097ab109d35b2a3da)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:18:27.789346 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d12354-196f-4aaf-b3b0-bbd29e97f909 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:18:27.821695 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 199/394] 10.222.0.49 () {58 vars in 1442 bytes} [Wed Jun 15 10:18:27 2022] PUT /identity/v3/projects/df1f045b58c24b4dba0964eff7d2e540/groups/77182ac3056944e19934f133b3f7b1b3/roles/6876e57cc51c448097ab109d35b2a3da => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:18:27.937281 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27380a84-1f07-4e58-89a7-b54fcfd96c39 None None] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:27.937577 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27380a84-1f07-4e58-89a7-b54fcfd96c39 None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:27.937758 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27380a84-1f07-4e58-89a7-b54fcfd96c39 None None] PATH_INFO: `/` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:27.939175 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 196/395] 10.222.0.49 () {54 vars in 926 bytes} [Wed Jun 15 10:18:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:18:27.954110 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5552fc-229d-47c3-9029-2b2bb17a7b48 None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:27.954604 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5552fc-229d-47c3-9029-2b2bb17a7b48 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:27.954677 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5552fc-229d-47c3-9029-2b2bb17a7b48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:27.984495 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-8b5552fc-229d-47c3-9029-2b2bb17a7b48 None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:18:27.985843 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-8b5552fc-229d-47c3-9029-2b2bb17a7b48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:18:28.056854 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671a83e1-d4d2-42b4-ac42-495fb1321002 None None] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:28.057802 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671a83e1-d4d2-42b4-ac42-495fb1321002 None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:28.058286 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671a83e1-d4d2-42b4-ac42-495fb1321002 None None] PATH_INFO: `/` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:28.060271 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 197/396] 10.222.0.49 () {54 vars in 926 bytes} [Wed Jun 15 10:18:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:18:28.065085 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3229aea8-a5de-4e60-85d4-5b49146a2b10 None None] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:28.065631 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3229aea8-a5de-4e60-85d4-5b49146a2b10 None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:28.066109 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3229aea8-a5de-4e60-85d4-5b49146a2b10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:28.081804 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 200/397] 10.222.0.49 () {58 vars in 1023 bytes} [Wed Jun 15 10:18:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 10:18:28.091439 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.auth.core [None req-3229aea8-a5de-4e60-85d4-5b49146a2b10 None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:18:28.092966 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-3229aea8-a5de-4e60-85d4-5b49146a2b10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:18:28.098841 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=100 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:18:28.119107 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5552fc-229d-47c3-9029-2b2bb17a7b48 None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:18:28.150689 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:18:28.219138 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 198/398] 10.222.0.49 () {58 vars in 1023 bytes} [Wed Jun 15 10:18:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 10:18:28.256530 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5552fc-229d-47c3-9029-2b2bb17a7b48 None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:18:28.259215 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3229aea8-a5de-4e60-85d4-5b49146a2b10 None None] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:18:28.307818 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb6b797-a366-4f24-8a09-09ac5a3e2634 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:18:28.308656 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb6b797-a366-4f24-8a09-09ac5a3e2634 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:28.308940 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb6b797-a366-4f24-8a09-09ac5a3e2634 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:28.309753 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb6b797-a366-4f24-8a09-09ac5a3e2634 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:28.328700 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb6b797-a366-4f24-8a09-09ac5a3e2634 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 10:18:28.350116 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application [None req-9fb6b797-a366-4f24-8a09-09ac5a3e2634 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 15 10:18:28.350116 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:18:28.350116 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:18:28.350116 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:18:28.350116 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:18:28.350116 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:18:28.350116 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:18:28.350116 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:18:28.350116 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:18:28.350116 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:18:28.350116 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:18:28.350116 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:18:28.350116 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 10:18:28.350116 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 10:18:28.350116 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 10:18:28.350116 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 10:18:28.351067 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 10:18:28.351067 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 10:18:28.351067 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 10:18:28.351067 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 10:18:28.351067 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:18:28.351067 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:18:28.351067 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:18:28.351067 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:18:28.351067 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:18:28.351067 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:18:28.351067 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:18:28.351067 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application with Lock( Jun 15 10:18:28.351067 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:18:28.351067 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:18:28.351067 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:18:28.351067 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:18:28.352013 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:18:28.352013 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:18:28.352013 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:18:28.352013 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:18:28.352013 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 10:18:28.352013 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 10:18:28.352013 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 10:18:28.352013 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 10:18:28.352013 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 10:18:28.352013 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 10:18:28.352013 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 15 10:18:28.352013 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Jun 15 10:18:28.352013 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 201/399] 10.222.0.49 () {56 vars in 1114 bytes} [Wed Jun 15 10:18:28 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:18:28.360014 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb6b797-a366-4f24-8a09-09ac5a3e2634 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:18:28.373022 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3229aea8-a5de-4e60-85d4-5b49146a2b10 None None] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:18:28.412835 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6990f085-e368-4851-b7b3-6a410088e4ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:18:28.414140 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6990f085-e368-4851-b7b3-6a410088e4ec None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:28.415417 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6990f085-e368-4851-b7b3-6a410088e4ec None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:28.416739 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6990f085-e368-4851-b7b3-6a410088e4ec None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:28.431306 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6990f085-e368-4851-b7b3-6a410088e4ec None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 10:18:28.439412 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=100 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:18:28.452306 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application [None req-6990f085-e368-4851-b7b3-6a410088e4ec None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 15 10:18:28.452306 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:18:28.452306 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:18:28.452306 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:18:28.452306 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:18:28.452306 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:18:28.452306 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:18:28.452306 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:18:28.452306 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:18:28.452306 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:18:28.452306 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:18:28.452306 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:18:28.452306 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 10:18:28.452306 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 10:18:28.452306 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 10:18:28.452306 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 10:18:28.453241 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 10:18:28.453241 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 10:18:28.453241 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 10:18:28.453241 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 10:18:28.453241 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:18:28.453241 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:18:28.453241 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:18:28.453241 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:18:28.453241 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:18:28.453241 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:18:28.453241 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:18:28.453241 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application with Lock( Jun 15 10:18:28.453241 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:18:28.453241 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:18:28.453241 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:18:28.453241 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:18:28.454130 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:18:28.454130 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:18:28.454130 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:18:28.454130 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:18:28.454130 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 10:18:28.454130 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 10:18:28.454130 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 10:18:28.454130 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 10:18:28.454130 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 10:18:28.454130 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 10:18:28.454130 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 15 10:18:28.454130 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application Jun 15 10:18:28.454130 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 199/400] 10.222.0.49 () {56 vars in 1114 bytes} [Wed Jun 15 10:18:28 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 10:18:28.461186 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6990f085-e368-4851-b7b3-6a410088e4ec None admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:18:28.500974 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:18:28.543323 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb6b797-a366-4f24-8a09-09ac5a3e2634 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:18:28.596515 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e39a02-66bb-4a25-b871-05c0a3a8f3cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:18:28.597204 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e39a02-66bb-4a25-b871-05c0a3a8f3cd None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:28.597493 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e39a02-66bb-4a25-b871-05c0a3a8f3cd None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:28.597772 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e39a02-66bb-4a25-b871-05c0a3a8f3cd None admin] PATH_INFO: `/v3/roles` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:28.598533 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e39a02-66bb-4a25-b871-05c0a3a8f3cd None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:18:28.598846 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e39a02-66bb-4a25-b871-05c0a3a8f3cd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:18:28.612008 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e39a02-66bb-4a25-b871-05c0a3a8f3cd None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:18:28.626663 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 202/401] 10.222.0.49 () {56 vars in 1111 bytes} [Wed Jun 15 10:18:28 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:18:28.630182 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e39a02-66bb-4a25-b871-05c0a3a8f3cd None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:18:28.657818 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6990f085-e368-4851-b7b3-6a410088e4ec None admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:18:28.714300 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a464f0-b6ec-4fa6-92a6-2a7ed2f62ee0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:18:28.716390 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a464f0-b6ec-4fa6-92a6-2a7ed2f62ee0 None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:28.718948 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a464f0-b6ec-4fa6-92a6-2a7ed2f62ee0 None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:28.719434 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a464f0-b6ec-4fa6-92a6-2a7ed2f62ee0 None admin] PATH_INFO: `/v3/roles` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:28.720426 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08a464f0-b6ec-4fa6-92a6-2a7ed2f62ee0 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:18:28.720871 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08a464f0-b6ec-4fa6-92a6-2a7ed2f62ee0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:18:28.724272 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08a464f0-b6ec-4fa6-92a6-2a7ed2f62ee0 None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:18:28.745063 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 200/402] 10.222.0.49 () {56 vars in 1111 bytes} [Wed Jun 15 10:18:28 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 10:18:28.752413 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a464f0-b6ec-4fa6-92a6-2a7ed2f62ee0 None admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:18:28.822327 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e39a02-66bb-4a25-b871-05c0a3a8f3cd None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:18:28.827867 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=100 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:18:28.861304 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3051bb88-5b7c-4f56-b672-93526577391c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:18:28.862531 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3051bb88-5b7c-4f56-b672-93526577391c None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:28.863043 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3051bb88-5b7c-4f56-b672-93526577391c None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:28.863490 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3051bb88-5b7c-4f56-b672-93526577391c None admin] PATH_INFO: `/v3/groups/77182ac3056944e19934f133b3f7b1b3` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:28.864867 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3051bb88-5b7c-4f56-b672-93526577391c None admin] RBAC: Authorizing `identity:get_group(group_id=77182ac3056944e19934f133b3f7b1b3)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:18:28.866700 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3051bb88-5b7c-4f56-b672-93526577391c None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:18:28.868995 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 203/403] 10.222.0.49 () {56 vars in 1180 bytes} [Wed Jun 15 10:18:28 2022] GET /identity/v3/groups/77182ac3056944e19934f133b3f7b1b3 => generated 226 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 10:18:28.870407 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422a94b0-5738-492c-b56f-705c76d4f7bc None None] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:28.870918 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422a94b0-5738-492c-b56f-705c76d4f7bc None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:28.871418 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422a94b0-5738-492c-b56f-705c76d4f7bc None None] PATH_INFO: `/` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:28.872492 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a464f0-b6ec-4fa6-92a6-2a7ed2f62ee0 None admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:18:28.877725 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 204/404] 10.222.0.49 () {54 vars in 926 bytes} [Wed Jun 15 10:18:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:18:28.878818 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422a94b0-5738-492c-b56f-705c76d4f7bc None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:18:28.891158 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:18:28.907608 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f046e01b-7d93-42e3-9b91-025abf77ff82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:18:28.908526 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f046e01b-7d93-42e3-9b91-025abf77ff82 None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:28.908847 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f046e01b-7d93-42e3-9b91-025abf77ff82 None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:28.909138 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f046e01b-7d93-42e3-9b91-025abf77ff82 None admin] PATH_INFO: `/v3/groups/77182ac3056944e19934f133b3f7b1b3` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:28.910325 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f046e01b-7d93-42e3-9b91-025abf77ff82 None admin] RBAC: Authorizing `identity:get_group(group_id=77182ac3056944e19934f133b3f7b1b3)` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:18:28.912037 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f046e01b-7d93-42e3-9b91-025abf77ff82 None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:18:28.914158 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 201/405] 10.222.0.49 () {56 vars in 1180 bytes} [Wed Jun 15 10:18:28 2022] GET /identity/v3/groups/77182ac3056944e19934f133b3f7b1b3 => generated 226 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 10:18:28.920665 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9961eb3-7e77-4a69-a694-dbd8e270b285 None None] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:28.921168 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9961eb3-7e77-4a69-a694-dbd8e270b285 None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:28.921743 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9961eb3-7e77-4a69-a694-dbd8e270b285 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:28.939765 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.auth.core [None req-b9961eb3-7e77-4a69-a694-dbd8e270b285 None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:18:28.941324 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-b9961eb3-7e77-4a69-a694-dbd8e270b285 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:18:28.993856 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422a94b0-5738-492c-b56f-705c76d4f7bc None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:18:29.011909 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 202/406] 10.222.0.49 () {58 vars in 1023 bytes} [Wed Jun 15 10:18:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 10:18:29.013043 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9961eb3-7e77-4a69-a694-dbd8e270b285 None None] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:18:29.038404 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf702e8c-3f4d-4cd2-bcfb-8b15a89c28f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:18:29.039369 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf702e8c-3f4d-4cd2-bcfb-8b15a89c28f4 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:29.039660 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf702e8c-3f4d-4cd2-bcfb-8b15a89c28f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:29.039944 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf702e8c-3f4d-4cd2-bcfb-8b15a89c28f4 None admin] PATH_INFO: `/v3/projects/5f3cd857995e418386bb22cdf90c1ae9` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:29.041372 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf702e8c-3f4d-4cd2-bcfb-8b15a89c28f4 None admin] RBAC: Authorizing `identity:get_project(project_id=5f3cd857995e418386bb22cdf90c1ae9)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:18:29.042760 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf702e8c-3f4d-4cd2-bcfb-8b15a89c28f4 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:18:29.045321 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 205/407] 10.222.0.49 () {56 vars in 1186 bytes} [Wed Jun 15 10:18:28 2022] GET /identity/v3/projects/5f3cd857995e418386bb22cdf90c1ae9 => generated 298 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:18:29.050888 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf702e8c-3f4d-4cd2-bcfb-8b15a89c28f4 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:18:29.077893 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=100 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:18:29.121199 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9961eb3-7e77-4a69-a694-dbd8e270b285 None None] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:18:29.136823 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:18:29.159812 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6223006-9b1b-43be-b8e2-e538b86cb0c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:18:29.161080 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6223006-9b1b-43be-b8e2-e538b86cb0c3 None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:29.161571 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6223006-9b1b-43be-b8e2-e538b86cb0c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:29.162082 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6223006-9b1b-43be-b8e2-e538b86cb0c3 None admin] PATH_INFO: `/v3/projects/df1f045b58c24b4dba0964eff7d2e540` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:29.163696 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf702e8c-3f4d-4cd2-bcfb-8b15a89c28f4 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:18:29.163941 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6223006-9b1b-43be-b8e2-e538b86cb0c3 None admin] RBAC: Authorizing `identity:get_project(project_id=df1f045b58c24b4dba0964eff7d2e540)` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:18:29.165664 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6223006-9b1b-43be-b8e2-e538b86cb0c3 None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:18:29.167800 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 203/408] 10.222.0.49 () {56 vars in 1186 bytes} [Wed Jun 15 10:18:29 2022] GET /identity/v3/projects/df1f045b58c24b4dba0964eff7d2e540 => generated 302 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:18:29.169425 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6223006-9b1b-43be-b8e2-e538b86cb0c3 None admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:18:29.206981 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8dae492-607d-4d7f-9c33-b1a7f61994c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:18:29.207921 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8dae492-607d-4d7f-9c33-b1a7f61994c1 None admin] REQUEST_METHOD: `PUT` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:29.208210 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8dae492-607d-4d7f-9c33-b1a7f61994c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:29.208492 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8dae492-607d-4d7f-9c33-b1a7f61994c1 None admin] PATH_INFO: `/v3/projects/5f3cd857995e418386bb22cdf90c1ae9/groups/77182ac3056944e19934f133b3f7b1b3/roles/4d2118569d074a288b37d120110c300d` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:29.211594 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8dae492-607d-4d7f-9c33-b1a7f61994c1 None admin] RBAC: Authorizing `identity:create_grant(project_id=5f3cd857995e418386bb22cdf90c1ae9, group_id=77182ac3056944e19934f133b3f7b1b3, role_id=4d2118569d074a288b37d120110c300d)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:18:29.213157 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8dae492-607d-4d7f-9c33-b1a7f61994c1 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:18:29.232314 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 206/409] 10.222.0.49 () {58 vars in 1442 bytes} [Wed Jun 15 10:18:29 2022] PUT /identity/v3/projects/5f3cd857995e418386bb22cdf90c1ae9/groups/77182ac3056944e19934f133b3f7b1b3/roles/4d2118569d074a288b37d120110c300d => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 10:18:29.234415 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8dae492-607d-4d7f-9c33-b1a7f61994c1 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:18:29.301902 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6223006-9b1b-43be-b8e2-e538b86cb0c3 None admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:18:29.330994 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=100 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:18:29.341761 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195c7680-ddad-4357-b4fa-e9bfdad08ab1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:18:29.342609 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195c7680-ddad-4357-b4fa-e9bfdad08ab1 None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:29.342897 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195c7680-ddad-4357-b4fa-e9bfdad08ab1 None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:29.347125 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195c7680-ddad-4357-b4fa-e9bfdad08ab1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:29.358127 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195c7680-ddad-4357-b4fa-e9bfdad08ab1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 10:18:29.366519 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application [None req-195c7680-ddad-4357-b4fa-e9bfdad08ab1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 10:18:29.366519 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:18:29.366519 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:18:29.366519 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:18:29.366519 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:18:29.366519 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:18:29.366519 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:18:29.366519 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:18:29.366519 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:18:29.366519 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:18:29.366519 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:18:29.366519 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:18:29.366519 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 10:18:29.366519 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 10:18:29.366519 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 10:18:29.366519 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 10:18:29.367666 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 10:18:29.367666 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 10:18:29.367666 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 10:18:29.367666 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 10:18:29.367666 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:18:29.367666 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:18:29.367666 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:18:29.367666 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:18:29.367666 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:18:29.367666 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:18:29.367666 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:18:29.367666 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application with Lock( Jun 15 10:18:29.367666 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:18:29.367666 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:18:29.367666 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:18:29.367666 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:18:29.368486 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:18:29.368486 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:18:29.368486 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:18:29.368486 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:18:29.368486 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 10:18:29.368486 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 10:18:29.368486 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 10:18:29.368486 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 10:18:29.368486 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 10:18:29.368486 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 10:18:29.368486 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 10:18:29.368486 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application Jun 15 10:18:29.368486 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 204/410] 10.222.0.49 () {56 vars in 1096 bytes} [Wed Jun 15 10:18:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:18:29.369396 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c430b753-c5f7-450a-a729-24c859dc8fe7 None None] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:29.369663 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c430b753-c5f7-450a-a729-24c859dc8fe7 None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:29.369946 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c430b753-c5f7-450a-a729-24c859dc8fe7 None None] PATH_INFO: `/` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:29.371519 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 205/411] 10.222.0.49 () {54 vars in 926 bytes} [Wed Jun 15 10:18:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:18:29.376638 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c430b753-c5f7-450a-a729-24c859dc8fe7 None None] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:18:29.385393 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8dae492-607d-4d7f-9c33-b1a7f61994c1 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:18:29.426267 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5993d2-9617-41d9-b04f-3a9ce01c4465 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:18:29.428203 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5993d2-9617-41d9-b04f-3a9ce01c4465 None admin] REQUEST_METHOD: `PUT` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:29.428490 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5993d2-9617-41d9-b04f-3a9ce01c4465 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:29.428767 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5993d2-9617-41d9-b04f-3a9ce01c4465 None admin] PATH_INFO: `/v3/projects/df1f045b58c24b4dba0964eff7d2e540/groups/77182ac3056944e19934f133b3f7b1b3/roles/4d2118569d074a288b37d120110c300d` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:29.431383 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b5993d2-9617-41d9-b04f-3a9ce01c4465 None admin] RBAC: Authorizing `identity:create_grant(project_id=df1f045b58c24b4dba0964eff7d2e540, group_id=77182ac3056944e19934f133b3f7b1b3, role_id=4d2118569d074a288b37d120110c300d)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:18:29.433738 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b5993d2-9617-41d9-b04f-3a9ce01c4465 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:18:29.437932 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:18:29.452279 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 207/412] 10.222.0.49 () {58 vars in 1442 bytes} [Wed Jun 15 10:18:29 2022] PUT /identity/v3/projects/df1f045b58c24b4dba0964eff7d2e540/groups/77182ac3056944e19934f133b3f7b1b3/roles/4d2118569d074a288b37d120110c300d => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:18:29.458126 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0351112a-59d7-474d-8a20-34c140319ef7 None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:29.458126 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0351112a-59d7-474d-8a20-34c140319ef7 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:29.458126 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0351112a-59d7-474d-8a20-34c140319ef7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:29.481000 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-0351112a-59d7-474d-8a20-34c140319ef7 None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:18:29.482246 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-0351112a-59d7-474d-8a20-34c140319ef7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:18:29.515972 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c430b753-c5f7-450a-a729-24c859dc8fe7 None None] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:18:29.574040 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 208/413] 10.222.0.49 () {58 vars in 1023 bytes} [Wed Jun 15 10:18:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 10:18:29.591949 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d683a88-f1bb-497c-9848-d8157acf39fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:18:29.592719 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d683a88-f1bb-497c-9848-d8157acf39fe None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:29.593006 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d683a88-f1bb-497c-9848-d8157acf39fe None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:29.593335 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d683a88-f1bb-497c-9848-d8157acf39fe None admin] PATH_INFO: `/v3/roles` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:29.594118 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d683a88-f1bb-497c-9848-d8157acf39fe None admin] RBAC: Adding query filter params (name=admin) {{(pid=80119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:18:29.594433 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d683a88-f1bb-497c-9848-d8157acf39fe None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:18:29.596097 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d683a88-f1bb-497c-9848-d8157acf39fe None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:18:29.611141 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 206/414] 10.222.0.49 () {56 vars in 1099 bytes} [Wed Jun 15 10:18:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:18:29.612420 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0351112a-59d7-474d-8a20-34c140319ef7 None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:18:29.622484 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d683a88-f1bb-497c-9848-d8157acf39fe None admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:18:29.701455 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=100 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:18:29.774327 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0351112a-59d7-474d-8a20-34c140319ef7 None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:18:29.785992 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d683a88-f1bb-497c-9848-d8157acf39fe None admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:18:29.816939 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:18:29.823127 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7c3759-b950-4e52-aa13-b05cf427571e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:18:29.823956 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7c3759-b950-4e52-aa13-b05cf427571e None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:29.824297 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7c3759-b950-4e52-aa13-b05cf427571e None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:29.824523 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7c3759-b950-4e52-aa13-b05cf427571e None admin] PATH_INFO: `/v3/roles/member` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:29.836619 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7c3759-b950-4e52-aa13-b05cf427571e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 10:18:29.843224 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c612d0a6-dd59-40d3-af60-372b92b55876 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:18:29.844115 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c612d0a6-dd59-40d3-af60-372b92b55876 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:29.844407 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c612d0a6-dd59-40d3-af60-372b92b55876 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:29.844690 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c612d0a6-dd59-40d3-af60-372b92b55876 None admin] PATH_INFO: `/v3/groups/ffc16ae2b8b843bc92af79734c11eeba` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:29.845284 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application [None req-ce7c3759-b950-4e52-aa13-b05cf427571e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 15 10:18:29.845284 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:18:29.845284 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:18:29.845284 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:18:29.845284 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:18:29.845284 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:18:29.845284 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:18:29.845284 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:18:29.845284 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:18:29.845284 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:18:29.845284 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:18:29.845284 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:18:29.845284 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 10:18:29.845284 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 10:18:29.845284 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 10:18:29.845284 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 10:18:29.846191 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 10:18:29.846191 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 10:18:29.846191 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 10:18:29.846191 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 10:18:29.846191 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:18:29.846191 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:18:29.846191 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:18:29.846191 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:18:29.846191 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:18:29.846191 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:18:29.846191 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:18:29.846191 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application with Lock( Jun 15 10:18:29.846191 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:18:29.846191 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:18:29.846191 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:18:29.846191 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:18:29.846989 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:18:29.846989 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:18:29.846989 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:18:29.846989 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:18:29.846989 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 10:18:29.846989 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 10:18:29.846989 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 10:18:29.846989 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 10:18:29.846989 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 10:18:29.846989 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 10:18:29.846989 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 15 10:18:29.846989 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application Jun 15 10:18:29.848778 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c612d0a6-dd59-40d3-af60-372b92b55876 None admin] RBAC: Authorizing `identity:get_group(group_id=ffc16ae2b8b843bc92af79734c11eeba)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:18:29.850455 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c612d0a6-dd59-40d3-af60-372b92b55876 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:18:29.851343 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 207/415] 10.222.0.49 () {56 vars in 1099 bytes} [Wed Jun 15 10:18:29 2022] GET /identity/v3/roles/member => generated 84 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:18:29.852867 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7c3759-b950-4e52-aa13-b05cf427571e None admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:18:29.852987 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 209/416] 10.222.0.49 () {56 vars in 1180 bytes} [Wed Jun 15 10:18:29 2022] GET /identity/v3/groups/ffc16ae2b8b843bc92af79734c11eeba => generated 229 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:18:29.860020 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c612d0a6-dd59-40d3-af60-372b92b55876 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:18:29.968761 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c612d0a6-dd59-40d3-af60-372b92b55876 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:18:29.991457 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7c3759-b950-4e52-aa13-b05cf427571e None admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:18:29.994719 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=100 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:18:30.025498 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f063bca1-5dc3-41cb-8028-2ef94f4a0a46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:18:30.026748 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f063bca1-5dc3-41cb-8028-2ef94f4a0a46 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:30.027040 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f063bca1-5dc3-41cb-8028-2ef94f4a0a46 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:30.027348 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f063bca1-5dc3-41cb-8028-2ef94f4a0a46 None admin] PATH_INFO: `/v3/projects/c8da8e1de64f469c9cc5f9a5f416060d` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:30.032701 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f063bca1-5dc3-41cb-8028-2ef94f4a0a46 None admin] RBAC: Authorizing `identity:get_project(project_id=c8da8e1de64f469c9cc5f9a5f416060d)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:18:30.035710 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f063bca1-5dc3-41cb-8028-2ef94f4a0a46 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:18:30.037474 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 210/417] 10.222.0.49 () {56 vars in 1186 bytes} [Wed Jun 15 10:18:29 2022] GET /identity/v3/projects/c8da8e1de64f469c9cc5f9a5f416060d => generated 344 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:18:30.039676 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e1288d-c118-4d35-888d-cdf9b2e1d5f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:18:30.040361 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e1288d-c118-4d35-888d-cdf9b2e1d5f9 None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:30.040646 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e1288d-c118-4d35-888d-cdf9b2e1d5f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:30.040925 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e1288d-c118-4d35-888d-cdf9b2e1d5f9 None admin] PATH_INFO: `/v3/roles` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:30.041734 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e1288d-c118-4d35-888d-cdf9b2e1d5f9 None admin] RBAC: Adding query filter params (name=member) {{(pid=80119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:18:30.042146 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e1288d-c118-4d35-888d-cdf9b2e1d5f9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:18:30.044123 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e1288d-c118-4d35-888d-cdf9b2e1d5f9 None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:18:30.046171 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f063bca1-5dc3-41cb-8028-2ef94f4a0a46 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:18:30.065476 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 208/418] 10.222.0.49 () {56 vars in 1101 bytes} [Wed Jun 15 10:18:29 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 10:18:30.070995 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e1288d-c118-4d35-888d-cdf9b2e1d5f9 None admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:18:30.167539 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:18:30.234458 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e1288d-c118-4d35-888d-cdf9b2e1d5f9 None admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:18:30.253280 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f063bca1-5dc3-41cb-8028-2ef94f4a0a46 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:18:30.270331 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d172f51-9e5d-4cf3-a55d-e1d2d5275cc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:18:30.271032 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d172f51-9e5d-4cf3-a55d-e1d2d5275cc6 None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:30.271390 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d172f51-9e5d-4cf3-a55d-e1d2d5275cc6 None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:30.271712 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d172f51-9e5d-4cf3-a55d-e1d2d5275cc6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:30.299106 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0601e4a2-bad2-4364-956a-71f2b3ace9d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:18:30.300305 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0601e4a2-bad2-4364-956a-71f2b3ace9d4 None admin] REQUEST_METHOD: `PUT` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:30.300876 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0601e4a2-bad2-4364-956a-71f2b3ace9d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:30.301605 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0601e4a2-bad2-4364-956a-71f2b3ace9d4 None admin] PATH_INFO: `/v3/projects/c8da8e1de64f469c9cc5f9a5f416060d/groups/ffc16ae2b8b843bc92af79734c11eeba/roles/8051174e7a7947d5b991ebd793c2e543` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:30.303828 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0601e4a2-bad2-4364-956a-71f2b3ace9d4 None admin] RBAC: Authorizing `identity:create_grant(project_id=c8da8e1de64f469c9cc5f9a5f416060d, group_id=ffc16ae2b8b843bc92af79734c11eeba, role_id=8051174e7a7947d5b991ebd793c2e543)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:18:30.305608 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0601e4a2-bad2-4364-956a-71f2b3ace9d4 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:18:30.315136 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=100 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:18:30.325490 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 211/419] 10.222.0.49 () {58 vars in 1442 bytes} [Wed Jun 15 10:18:30 2022] PUT /identity/v3/projects/c8da8e1de64f469c9cc5f9a5f416060d/groups/ffc16ae2b8b843bc92af79734c11eeba/roles/8051174e7a7947d5b991ebd793c2e543 => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:18:30.327289 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a15d50-ec6f-4ca7-905f-9ecd79cd0ff9 None None] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:30.327800 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a15d50-ec6f-4ca7-905f-9ecd79cd0ff9 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:30.328269 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a15d50-ec6f-4ca7-905f-9ecd79cd0ff9 None None] PATH_INFO: `/` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:30.335484 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 212/420] 10.222.0.49 () {54 vars in 926 bytes} [Wed Jun 15 10:18:30 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:18:30.344846 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651612ef-b646-408e-b82c-c861bd3b4a72 None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:30.345440 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651612ef-b646-408e-b82c-c861bd3b4a72 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:30.345950 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651612ef-b646-408e-b82c-c861bd3b4a72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:30.349439 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d172f51-9e5d-4cf3-a55d-e1d2d5275cc6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:18:30.351033 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d172f51-9e5d-4cf3-a55d-e1d2d5275cc6 None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:18:30.374983 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-651612ef-b646-408e-b82c-c861bd3b4a72 None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:18:30.376625 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-651612ef-b646-408e-b82c-c861bd3b4a72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:18:30.467943 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 209/421] 10.222.0.49 () {58 vars in 1282 bytes} [Wed Jun 15 10:18:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 397 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 15 10:18:30.480238 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d172f51-9e5d-4cf3-a55d-e1d2d5275cc6 None admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:18:30.489597 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 213/422] 10.222.0.49 () {58 vars in 1023 bytes} [Wed Jun 15 10:18:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Jun 15 10:18:30.521453 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651612ef-b646-408e-b82c-c861bd3b4a72 None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:18:30.598474 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=100 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:18:30.631166 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:18:30.655997 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d172f51-9e5d-4cf3-a55d-e1d2d5275cc6 None admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:18:30.699019 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ca7ff3-fa3f-46ff-8815-cfa4668b4a6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:18:30.700294 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ca7ff3-fa3f-46ff-8815-cfa4668b4a6e None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:30.700814 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ca7ff3-fa3f-46ff-8815-cfa4668b4a6e None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:30.701321 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ca7ff3-fa3f-46ff-8815-cfa4668b4a6e None admin] PATH_INFO: `/v3/projects/5f3cd857995e418386bb22cdf90c1ae9` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:30.702760 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ca7ff3-fa3f-46ff-8815-cfa4668b4a6e None admin] RBAC: Authorizing `identity:get_project(project_id=5f3cd857995e418386bb22cdf90c1ae9)` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:18:30.703053 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651612ef-b646-408e-b82c-c861bd3b4a72 None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:18:30.704719 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ca7ff3-fa3f-46ff-8815-cfa4668b4a6e None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:18:30.711404 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 210/423] 10.222.0.49 () {56 vars in 1186 bytes} [Wed Jun 15 10:18:30 2022] GET /identity/v3/projects/5f3cd857995e418386bb22cdf90c1ae9 => generated 298 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:18:30.712835 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ca7ff3-fa3f-46ff-8815-cfa4668b4a6e None admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:18:30.748203 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796ba765-f74e-4092-9f6b-3370a1b1011e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:18:30.749159 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796ba765-f74e-4092-9f6b-3370a1b1011e None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:30.749582 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796ba765-f74e-4092-9f6b-3370a1b1011e None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:30.749873 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796ba765-f74e-4092-9f6b-3370a1b1011e None admin] PATH_INFO: `/v3/roles/member` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:30.762385 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-796ba765-f74e-4092-9f6b-3370a1b1011e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 10:18:30.774167 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application [None req-796ba765-f74e-4092-9f6b-3370a1b1011e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 15 10:18:30.774167 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:18:30.774167 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:18:30.774167 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:18:30.774167 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:18:30.774167 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:18:30.774167 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:18:30.774167 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:18:30.774167 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:18:30.774167 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:18:30.774167 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:18:30.774167 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:18:30.774167 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 10:18:30.774167 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 10:18:30.774167 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 10:18:30.774167 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 10:18:30.775058 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 10:18:30.775058 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 10:18:30.775058 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 10:18:30.775058 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 10:18:30.775058 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:18:30.775058 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:18:30.775058 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:18:30.775058 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:18:30.775058 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:18:30.775058 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:18:30.775058 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:18:30.775058 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application with Lock( Jun 15 10:18:30.775058 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:18:30.775058 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:18:30.775058 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:18:30.775058 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:18:30.775887 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:18:30.775887 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:18:30.775887 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:18:30.775887 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:18:30.775887 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 10:18:30.775887 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 10:18:30.775887 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 10:18:30.775887 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 10:18:30.775887 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 10:18:30.775887 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 10:18:30.775887 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 15 10:18:30.775887 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Jun 15 10:18:30.779228 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 214/424] 10.222.0.49 () {56 vars in 1099 bytes} [Wed Jun 15 10:18:30 2022] GET /identity/v3/roles/member => generated 84 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:18:30.781309 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796ba765-f74e-4092-9f6b-3370a1b1011e None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:18:30.846535 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ca7ff3-fa3f-46ff-8815-cfa4668b4a6e None admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:18:30.912616 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796ba765-f74e-4092-9f6b-3370a1b1011e None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:18:30.917461 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e072bbd-fae2-412b-85d2-88a1f8f5aac6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:18:30.918778 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e072bbd-fae2-412b-85d2-88a1f8f5aac6 None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:30.919318 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e072bbd-fae2-412b-85d2-88a1f8f5aac6 None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:30.919844 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e072bbd-fae2-412b-85d2-88a1f8f5aac6 None admin] PATH_INFO: `/v3/groups/77182ac3056944e19934f133b3f7b1b3` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:30.927443 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:18:30.928130 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e072bbd-fae2-412b-85d2-88a1f8f5aac6 None admin] RBAC: Authorizing `identity:get_group(group_id=77182ac3056944e19934f133b3f7b1b3)` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:18:30.930103 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e072bbd-fae2-412b-85d2-88a1f8f5aac6 None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:18:30.932294 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 211/425] 10.222.0.49 () {56 vars in 1180 bytes} [Wed Jun 15 10:18:30 2022] GET /identity/v3/groups/77182ac3056944e19934f133b3f7b1b3 => generated 226 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 10:18:30.938695 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e072bbd-fae2-412b-85d2-88a1f8f5aac6 None admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:18:30.952812 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=100 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:18:30.957413 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b82859-0815-4265-8e5d-00ae3bd85494 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:18:30.960157 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b82859-0815-4265-8e5d-00ae3bd85494 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:30.960593 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b82859-0815-4265-8e5d-00ae3bd85494 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:30.962448 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b82859-0815-4265-8e5d-00ae3bd85494 None admin] PATH_INFO: `/v3/roles` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:30.963528 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b82859-0815-4265-8e5d-00ae3bd85494 None admin] RBAC: Adding query filter params (name=member) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:18:30.963987 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b82859-0815-4265-8e5d-00ae3bd85494 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:18:30.966352 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b82859-0815-4265-8e5d-00ae3bd85494 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:18:30.981097 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 215/426] 10.222.0.49 () {56 vars in 1101 bytes} [Wed Jun 15 10:18:30 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 10:18:30.982630 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b82859-0815-4265-8e5d-00ae3bd85494 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:18:31.108053 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e072bbd-fae2-412b-85d2-88a1f8f5aac6 None admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:18:31.159129 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b82859-0815-4265-8e5d-00ae3bd85494 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:18:31.183807 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4e5d03-a645-4a0e-9ed6-98d68ec72098 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:18:31.184638 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4e5d03-a645-4a0e-9ed6-98d68ec72098 None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:31.185014 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4e5d03-a645-4a0e-9ed6-98d68ec72098 None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:31.185384 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4e5d03-a645-4a0e-9ed6-98d68ec72098 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:31.186432 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd4e5d03-a645-4a0e-9ed6-98d68ec72098 None admin] RBAC: Adding query filter params (group.id=77182ac3056944e19934f133b3f7b1b3, role.id=6876e57cc51c448097ab109d35b2a3da, scope.project.id=5f3cd857995e418386bb22cdf90c1ae9) {{(pid=80119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:18:31.186939 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd4e5d03-a645-4a0e-9ed6-98d68ec72098 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:18:31.188497 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd4e5d03-a645-4a0e-9ed6-98d68ec72098 None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:18:31.202209 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4fec7c6-c9a2-4eb5-97c8-a3589603a67d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:18:31.203489 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 212/427] 10.222.0.49 () {56 vars in 1376 bytes} [Wed Jun 15 10:18:30 2022] GET /identity/v3/role_assignments?group.id=77182ac3056944e19934f133b3f7b1b3&scope.project.id=5f3cd857995e418386bb22cdf90c1ae9&role.id=6876e57cc51c448097ab109d35b2a3da => generated 611 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 10:18:31.204392 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fec7c6-c9a2-4eb5-97c8-a3589603a67d None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:31.204945 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fec7c6-c9a2-4eb5-97c8-a3589603a67d None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:31.205368 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fec7c6-c9a2-4eb5-97c8-a3589603a67d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:31.259503 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:18:31.281841 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4fec7c6-c9a2-4eb5-97c8-a3589603a67d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:18:31.283586 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4fec7c6-c9a2-4eb5-97c8-a3589603a67d None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:18:31.392005 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 216/428] 10.222.0.49 () {58 vars in 1282 bytes} [Wed Jun 15 10:18:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 409 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 15 10:18:31.400301 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4e5d03-a645-4a0e-9ed6-98d68ec72098 None admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:18:31.484602 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=100 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:18:31.520992 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4e5d03-a645-4a0e-9ed6-98d68ec72098 None admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:18:31.557527 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08672326-ed2d-4e29-9186-cde4d4eaa056 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:18:31.558674 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08672326-ed2d-4e29-9186-cde4d4eaa056 None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:31.559704 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08672326-ed2d-4e29-9186-cde4d4eaa056 None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:31.560128 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08672326-ed2d-4e29-9186-cde4d4eaa056 None admin] PATH_INFO: `/v3/projects/df1f045b58c24b4dba0964eff7d2e540` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:31.561495 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08672326-ed2d-4e29-9186-cde4d4eaa056 None admin] RBAC: Authorizing `identity:get_project(project_id=df1f045b58c24b4dba0964eff7d2e540)` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:18:31.563134 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08672326-ed2d-4e29-9186-cde4d4eaa056 None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:18:31.565319 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 213/429] 10.222.0.49 () {56 vars in 1186 bytes} [Wed Jun 15 10:18:31 2022] GET /identity/v3/projects/df1f045b58c24b4dba0964eff7d2e540 => generated 302 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:18:31.570660 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4fec7c6-c9a2-4eb5-97c8-a3589603a67d None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:18:31.651718 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b28dd42-d74f-40e2-a962-e8403ba45dc9 None None] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:31.651967 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b28dd42-d74f-40e2-a962-e8403ba45dc9 None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:31.652259 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b28dd42-d74f-40e2-a962-e8403ba45dc9 None None] PATH_INFO: `/` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:31.653685 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 214/430] 10.222.0.49 () {54 vars in 926 bytes} [Wed Jun 15 10:18:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 15 10:18:31.656622 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:18:31.658983 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c0f87c-273b-4cd0-b7d7-8dd8c904064d None None] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:31.659298 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c0f87c-273b-4cd0-b7d7-8dd8c904064d None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:31.659603 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c0f87c-273b-4cd0-b7d7-8dd8c904064d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:31.681902 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.auth.core [None req-a8c0f87c-273b-4cd0-b7d7-8dd8c904064d None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:18:31.683483 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-a8c0f87c-273b-4cd0-b7d7-8dd8c904064d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:18:31.686667 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4fec7c6-c9a2-4eb5-97c8-a3589603a67d None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:18:31.731441 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1271ca56-b166-455d-ba3f-0a26c429e366 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:18:31.733080 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1271ca56-b166-455d-ba3f-0a26c429e366 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:31.733515 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1271ca56-b166-455d-ba3f-0a26c429e366 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:31.733938 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1271ca56-b166-455d-ba3f-0a26c429e366 None admin] PATH_INFO: `/v3/groups/77182ac3056944e19934f133b3f7b1b3` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:31.735388 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1271ca56-b166-455d-ba3f-0a26c429e366 None admin] RBAC: Authorizing `identity:get_group(group_id=77182ac3056944e19934f133b3f7b1b3)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:18:31.737717 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1271ca56-b166-455d-ba3f-0a26c429e366 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:18:31.740397 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 217/431] 10.222.0.49 () {56 vars in 1180 bytes} [Wed Jun 15 10:18:31 2022] GET /identity/v3/groups/77182ac3056944e19934f133b3f7b1b3 => generated 226 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 10:18:31.747966 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1271ca56-b166-455d-ba3f-0a26c429e366 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:18:31.770664 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 215/432] 10.222.0.49 () {58 vars in 1023 bytes} [Wed Jun 15 10:18:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 15 10:18:31.814579 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c0f87c-273b-4cd0-b7d7-8dd8c904064d None None] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:18:31.877680 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1271ca56-b166-455d-ba3f-0a26c429e366 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:18:31.896390 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=100 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:18:31.927758 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa46e4c6-c376-4c49-98b6-2b01f591da13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:18:31.928653 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa46e4c6-c376-4c49-98b6-2b01f591da13 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:31.928947 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa46e4c6-c376-4c49-98b6-2b01f591da13 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:31.929229 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa46e4c6-c376-4c49-98b6-2b01f591da13 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:31.930110 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa46e4c6-c376-4c49-98b6-2b01f591da13 None admin] RBAC: Adding query filter params (group.id=77182ac3056944e19934f133b3f7b1b3, role.id=6876e57cc51c448097ab109d35b2a3da, scope.project.id=df1f045b58c24b4dba0964eff7d2e540) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:18:31.930419 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa46e4c6-c376-4c49-98b6-2b01f591da13 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:18:31.932797 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa46e4c6-c376-4c49-98b6-2b01f591da13 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:18:31.947114 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 218/433] 10.222.0.49 () {56 vars in 1376 bytes} [Wed Jun 15 10:18:31 2022] GET /identity/v3/role_assignments?group.id=77182ac3056944e19934f133b3f7b1b3&scope.project.id=df1f045b58c24b4dba0964eff7d2e540&role.id=6876e57cc51c448097ab109d35b2a3da => generated 611 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:18:31.954933 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c0f87c-273b-4cd0-b7d7-8dd8c904064d None None] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:18:32.000658 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3df0cb-2c92-49da-a11b-ec09e82b09c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:18:32.001510 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3df0cb-2c92-49da-a11b-ec09e82b09c5 None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:32.001798 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3df0cb-2c92-49da-a11b-ec09e82b09c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:32.002076 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3df0cb-2c92-49da-a11b-ec09e82b09c5 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:32.015694 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd3df0cb-2c92-49da-a11b-ec09e82b09c5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 10:18:32.026905 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application [None req-fd3df0cb-2c92-49da-a11b-ec09e82b09c5 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 15 10:18:32.026905 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:18:32.026905 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:18:32.026905 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:18:32.026905 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:18:32.026905 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:18:32.026905 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:18:32.026905 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:18:32.026905 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:18:32.026905 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:18:32.026905 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:18:32.026905 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:18:32.026905 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 10:18:32.026905 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 10:18:32.026905 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 10:18:32.026905 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 10:18:32.028439 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 10:18:32.028439 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 10:18:32.028439 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 10:18:32.028439 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 10:18:32.028439 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:18:32.028439 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:18:32.028439 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:18:32.028439 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:18:32.028439 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:18:32.028439 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:18:32.028439 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:18:32.028439 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application with Lock( Jun 15 10:18:32.028439 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:18:32.028439 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:18:32.028439 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:18:32.028439 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:18:32.029242 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:18:32.029242 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:18:32.029242 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:18:32.029242 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:18:32.029242 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 10:18:32.029242 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 10:18:32.029242 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 10:18:32.029242 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 10:18:32.029242 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 10:18:32.029242 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 10:18:32.029242 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 15 10:18:32.029242 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application Jun 15 10:18:32.029242 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 216/434] 10.222.0.49 () {56 vars in 1114 bytes} [Wed Jun 15 10:18:31 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 10:18:32.039449 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa46e4c6-c376-4c49-98b6-2b01f591da13 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:18:32.061167 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:18:32.151177 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f429b99-7870-4f4a-b90c-bb6ce3937cf0 None None] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:32.151481 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f429b99-7870-4f4a-b90c-bb6ce3937cf0 None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:32.151738 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f429b99-7870-4f4a-b90c-bb6ce3937cf0 None None] PATH_INFO: `/` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:32.153787 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 217/435] 10.222.0.49 () {54 vars in 926 bytes} [Wed Jun 15 10:18:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 15 10:18:32.159131 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858a6ef3-b4b3-4435-9802-0c5d4f627e7c None None] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:32.159412 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858a6ef3-b4b3-4435-9802-0c5d4f627e7c None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:32.159698 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858a6ef3-b4b3-4435-9802-0c5d4f627e7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:32.181829 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa46e4c6-c376-4c49-98b6-2b01f591da13 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:18:32.186371 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.auth.core [None req-858a6ef3-b4b3-4435-9802-0c5d4f627e7c None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:18:32.187686 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-858a6ef3-b4b3-4435-9802-0c5d4f627e7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:18:32.225896 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf6d596-b6bb-48e4-a13a-704e1149022b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:18:32.226871 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf6d596-b6bb-48e4-a13a-704e1149022b None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:32.227210 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf6d596-b6bb-48e4-a13a-704e1149022b None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:32.227505 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf6d596-b6bb-48e4-a13a-704e1149022b None admin] PATH_INFO: `/v3/roles` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:32.228535 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf6d596-b6bb-48e4-a13a-704e1149022b None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:18:32.229238 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf6d596-b6bb-48e4-a13a-704e1149022b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:18:32.231564 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf6d596-b6bb-48e4-a13a-704e1149022b None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:18:32.242869 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 219/436] 10.222.0.49 () {56 vars in 1111 bytes} [Wed Jun 15 10:18:32 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:18:32.249886 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf6d596-b6bb-48e4-a13a-704e1149022b None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:18:32.275225 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 218/437] 10.222.0.49 () {58 vars in 1023 bytes} [Wed Jun 15 10:18:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 10:18:32.297662 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858a6ef3-b4b3-4435-9802-0c5d4f627e7c None None] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:18:32.319806 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:18:32.357730 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf6d596-b6bb-48e4-a13a-704e1149022b None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:18:32.364887 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=100 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:18:32.396070 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d68462-5807-4238-9679-2659bce1e066 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:18:32.396734 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d68462-5807-4238-9679-2659bce1e066 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:32.397021 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d68462-5807-4238-9679-2659bce1e066 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:32.397304 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d68462-5807-4238-9679-2659bce1e066 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:32.410040 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858a6ef3-b4b3-4435-9802-0c5d4f627e7c None None] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:18:32.446892 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b965be2-2758-473f-b0bd-17497aeae920 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:18:32.448023 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b965be2-2758-473f-b0bd-17497aeae920 None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:32.448382 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b965be2-2758-473f-b0bd-17497aeae920 None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:32.448648 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b965be2-2758-473f-b0bd-17497aeae920 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:32.459023 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b965be2-2758-473f-b0bd-17497aeae920 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 10:18:32.467178 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application [None req-3b965be2-2758-473f-b0bd-17497aeae920 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 15 10:18:32.467178 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:18:32.467178 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:18:32.467178 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:18:32.467178 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:18:32.467178 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:18:32.467178 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:18:32.467178 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:18:32.467178 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:18:32.467178 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:18:32.467178 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:18:32.467178 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:18:32.467178 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 10:18:32.467178 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 10:18:32.467178 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 10:18:32.467178 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 10:18:32.468291 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 10:18:32.468291 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 10:18:32.468291 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 10:18:32.468291 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 10:18:32.468291 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:18:32.468291 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:18:32.468291 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:18:32.468291 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:18:32.468291 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:18:32.468291 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:18:32.468291 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:18:32.468291 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application with Lock( Jun 15 10:18:32.468291 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:18:32.468291 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:18:32.468291 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:18:32.468291 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:18:32.469241 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:18:32.469241 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:18:32.469241 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:18:32.469241 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:18:32.469241 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 10:18:32.469241 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 10:18:32.469241 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 10:18:32.469241 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 10:18:32.469241 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 10:18:32.469241 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 10:18:32.469241 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 15 10:18:32.469241 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application Jun 15 10:18:32.469241 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 219/438] 10.222.0.49 () {56 vars in 1114 bytes} [Wed Jun 15 10:18:32 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 10:18:32.472303 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d68462-5807-4238-9679-2659bce1e066 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:18:32.473833 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b965be2-2758-473f-b0bd-17497aeae920 None admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:18:32.473938 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d68462-5807-4238-9679-2659bce1e066 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:18:32.543954 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:18:32.580372 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 220/439] 10.222.0.49 () {58 vars in 1282 bytes} [Wed Jun 15 10:18:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 15 10:18:32.581095 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b965be2-2758-473f-b0bd-17497aeae920 None admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:18:32.585724 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d68462-5807-4238-9679-2659bce1e066 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:18:32.598213 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=100 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:18:32.617252 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5366463-a253-4dc8-8857-60388bd8aba7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:18:32.617927 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5366463-a253-4dc8-8857-60388bd8aba7 None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:32.618315 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5366463-a253-4dc8-8857-60388bd8aba7 None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:32.618591 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5366463-a253-4dc8-8857-60388bd8aba7 None admin] PATH_INFO: `/v3/roles` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:32.619513 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5366463-a253-4dc8-8857-60388bd8aba7 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:18:32.619797 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5366463-a253-4dc8-8857-60388bd8aba7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:18:32.621188 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5366463-a253-4dc8-8857-60388bd8aba7 None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:18:32.634160 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 220/440] 10.222.0.49 () {56 vars in 1111 bytes} [Wed Jun 15 10:18:32 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 10:18:32.639906 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5366463-a253-4dc8-8857-60388bd8aba7 None admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:18:32.694913 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d68462-5807-4238-9679-2659bce1e066 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:18:32.729046 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08fb0dc1-c5e1-4c34-b408-3449d883782c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:18:32.729833 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fb0dc1-c5e1-4c34-b408-3449d883782c None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:32.730117 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fb0dc1-c5e1-4c34-b408-3449d883782c None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:32.730398 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fb0dc1-c5e1-4c34-b408-3449d883782c None admin] PATH_INFO: `/v3/projects/5f3cd857995e418386bb22cdf90c1ae9` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:32.732118 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08fb0dc1-c5e1-4c34-b408-3449d883782c None admin] RBAC: Authorizing `identity:get_project(project_id=5f3cd857995e418386bb22cdf90c1ae9)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:18:32.733554 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08fb0dc1-c5e1-4c34-b408-3449d883782c None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:18:32.735283 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 221/441] 10.222.0.49 () {56 vars in 1186 bytes} [Wed Jun 15 10:18:32 2022] GET /identity/v3/projects/5f3cd857995e418386bb22cdf90c1ae9 => generated 298 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 10:18:32.740521 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08fb0dc1-c5e1-4c34-b408-3449d883782c None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:18:32.761341 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5366463-a253-4dc8-8857-60388bd8aba7 None admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:18:32.779595 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:18:32.800984 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41292f4e-ca17-4724-915c-d2cfe08dbcfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:18:32.801904 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41292f4e-ca17-4724-915c-d2cfe08dbcfd None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:32.802190 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41292f4e-ca17-4724-915c-d2cfe08dbcfd None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:32.802469 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41292f4e-ca17-4724-915c-d2cfe08dbcfd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:32.853219 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08fb0dc1-c5e1-4c34-b408-3449d883782c None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:18:32.858404 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=100 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:18:32.879858 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41292f4e-ca17-4724-915c-d2cfe08dbcfd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:18:32.881645 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41292f4e-ca17-4724-915c-d2cfe08dbcfd None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:18:32.890769 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5008d7-f837-422e-b9a8-b878493fce8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:18:32.892010 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5008d7-f837-422e-b9a8-b878493fce8b None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:32.892303 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5008d7-f837-422e-b9a8-b878493fce8b None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:32.892580 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5008d7-f837-422e-b9a8-b878493fce8b None admin] PATH_INFO: `/v3/groups/77182ac3056944e19934f133b3f7b1b3` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:32.894052 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f5008d7-f837-422e-b9a8-b878493fce8b None admin] RBAC: Authorizing `identity:get_group(group_id=77182ac3056944e19934f133b3f7b1b3)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:18:32.895518 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f5008d7-f837-422e-b9a8-b878493fce8b None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:18:32.897298 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 222/442] 10.222.0.49 () {56 vars in 1180 bytes} [Wed Jun 15 10:18:32 2022] GET /identity/v3/groups/77182ac3056944e19934f133b3f7b1b3 => generated 226 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 10:18:32.903449 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5008d7-f837-422e-b9a8-b878493fce8b None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:18:32.990338 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 221/443] 10.222.0.49 () {58 vars in 1282 bytes} [Wed Jun 15 10:18:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 15 10:18:32.995890 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41292f4e-ca17-4724-915c-d2cfe08dbcfd None admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:18:33.019333 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5008d7-f837-422e-b9a8-b878493fce8b None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:18:33.022690 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:18:33.055825 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c5c1a0-75c9-423f-a9c0-d223a49e11db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:18:33.056480 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c5c1a0-75c9-423f-a9c0-d223a49e11db None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:33.056761 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c5c1a0-75c9-423f-a9c0-d223a49e11db None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:33.057034 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c5c1a0-75c9-423f-a9c0-d223a49e11db None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:33.057851 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c5c1a0-75c9-423f-a9c0-d223a49e11db None admin] RBAC: Adding query filter params (group.id=77182ac3056944e19934f133b3f7b1b3, role.id=4d2118569d074a288b37d120110c300d, scope.project.id=5f3cd857995e418386bb22cdf90c1ae9) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:18:33.058163 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c5c1a0-75c9-423f-a9c0-d223a49e11db None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:18:33.059749 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c5c1a0-75c9-423f-a9c0-d223a49e11db None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:18:33.065836 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 223/444] 10.222.0.49 () {56 vars in 1376 bytes} [Wed Jun 15 10:18:32 2022] GET /identity/v3/role_assignments?group.id=77182ac3056944e19934f133b3f7b1b3&scope.project.id=5f3cd857995e418386bb22cdf90c1ae9&role.id=4d2118569d074a288b37d120110c300d => generated 611 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 10:18:33.067019 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39ab631-d041-47e6-b0a7-e3983b3fd87e None None] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:33.067331 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39ab631-d041-47e6-b0a7-e3983b3fd87e None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:33.067607 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39ab631-d041-47e6-b0a7-e3983b3fd87e None None] PATH_INFO: `/` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:33.068994 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 224/445] 10.222.0.49 () {54 vars in 926 bytes} [Wed Jun 15 10:18:33 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:18:33.074525 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecbff05-109e-4eaf-b12e-928f9048d41e None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:33.074826 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecbff05-109e-4eaf-b12e-928f9048d41e None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:33.075114 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecbff05-109e-4eaf-b12e-928f9048d41e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:33.080092 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=100 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:18:33.093151 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-aecbff05-109e-4eaf-b12e-928f9048d41e None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:18:33.094849 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-aecbff05-109e-4eaf-b12e-928f9048d41e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:18:33.101914 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41292f4e-ca17-4724-915c-d2cfe08dbcfd None admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:18:33.141960 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89fc284-1a3f-4608-8a7d-1c3a238576c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:18:33.142956 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89fc284-1a3f-4608-8a7d-1c3a238576c0 None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:33.143281 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89fc284-1a3f-4608-8a7d-1c3a238576c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:33.143557 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89fc284-1a3f-4608-8a7d-1c3a238576c0 None admin] PATH_INFO: `/v3/projects/df1f045b58c24b4dba0964eff7d2e540` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:33.144807 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f89fc284-1a3f-4608-8a7d-1c3a238576c0 None admin] RBAC: Authorizing `identity:get_project(project_id=df1f045b58c24b4dba0964eff7d2e540)` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:18:33.146439 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f89fc284-1a3f-4608-8a7d-1c3a238576c0 None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:18:33.148238 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 222/446] 10.222.0.49 () {56 vars in 1186 bytes} [Wed Jun 15 10:18:32 2022] GET /identity/v3/projects/df1f045b58c24b4dba0964eff7d2e540 => generated 302 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 10:18:33.154433 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89fc284-1a3f-4608-8a7d-1c3a238576c0 None admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:18:33.174250 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 225/447] 10.222.0.49 () {58 vars in 1023 bytes} [Wed Jun 15 10:18:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 15 10:18:33.197390 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aecbff05-109e-4eaf-b12e-928f9048d41e None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:18:33.270458 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89fc284-1a3f-4608-8a7d-1c3a238576c0 None admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:18:33.304718 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:18:33.310497 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aecbff05-109e-4eaf-b12e-928f9048d41e None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:18:33.314622 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0835d7ba-d46a-4ecb-87ca-6895706b373e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:18:33.315573 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0835d7ba-d46a-4ecb-87ca-6895706b373e None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:33.316905 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0835d7ba-d46a-4ecb-87ca-6895706b373e None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:33.316905 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0835d7ba-d46a-4ecb-87ca-6895706b373e None admin] PATH_INFO: `/v3/groups/77182ac3056944e19934f133b3f7b1b3` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:33.317298 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0835d7ba-d46a-4ecb-87ca-6895706b373e None admin] RBAC: Authorizing `identity:get_group(group_id=77182ac3056944e19934f133b3f7b1b3)` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:18:33.318764 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0835d7ba-d46a-4ecb-87ca-6895706b373e None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:18:33.320438 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 223/448] 10.222.0.49 () {56 vars in 1180 bytes} [Wed Jun 15 10:18:33 2022] GET /identity/v3/groups/77182ac3056944e19934f133b3f7b1b3 => generated 226 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 10:18:33.328002 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0835d7ba-d46a-4ecb-87ca-6895706b373e None admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:18:33.351683 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=100 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:18:33.359618 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880ab57f-ef78-42bd-b724-80562f5ed7ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:18:33.360747 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880ab57f-ef78-42bd-b724-80562f5ed7ba None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:33.361035 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880ab57f-ef78-42bd-b724-80562f5ed7ba None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:33.361455 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880ab57f-ef78-42bd-b724-80562f5ed7ba None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:33.374736 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-880ab57f-ef78-42bd-b724-80562f5ed7ba None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 10:18:33.383268 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application [None req-880ab57f-ef78-42bd-b724-80562f5ed7ba None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 10:18:33.383268 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:18:33.383268 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:18:33.383268 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:18:33.383268 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:18:33.383268 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:18:33.383268 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:18:33.383268 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:18:33.383268 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:18:33.383268 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:18:33.383268 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:18:33.383268 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:18:33.383268 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 10:18:33.383268 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 10:18:33.383268 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 10:18:33.383268 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 10:18:33.384166 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 10:18:33.384166 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 10:18:33.384166 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 10:18:33.384166 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 10:18:33.384166 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:18:33.384166 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:18:33.384166 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:18:33.384166 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:18:33.384166 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:18:33.384166 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:18:33.384166 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:18:33.384166 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application with Lock( Jun 15 10:18:33.384166 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:18:33.384166 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:18:33.384166 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:18:33.384166 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:18:33.384965 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:18:33.384965 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:18:33.384965 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:18:33.384965 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:18:33.384965 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 10:18:33.384965 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 10:18:33.384965 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 10:18:33.384965 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 10:18:33.384965 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 10:18:33.384965 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 10:18:33.384965 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 10:18:33.384965 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Jun 15 10:18:33.389424 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 226/449] 10.222.0.49 () {56 vars in 1096 bytes} [Wed Jun 15 10:18:33 2022] GET /identity/v3/roles/admin => generated 83 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 10:18:33.390787 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880ab57f-ef78-42bd-b724-80562f5ed7ba None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:18:33.454135 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0835d7ba-d46a-4ecb-87ca-6895706b373e None admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:18:33.494497 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de1d7e8-1b06-4433-b0e9-199728a1a054 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:18:33.495197 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de1d7e8-1b06-4433-b0e9-199728a1a054 None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:33.495487 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de1d7e8-1b06-4433-b0e9-199728a1a054 None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:33.495762 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de1d7e8-1b06-4433-b0e9-199728a1a054 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:33.496633 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de1d7e8-1b06-4433-b0e9-199728a1a054 None admin] RBAC: Adding query filter params (group.id=77182ac3056944e19934f133b3f7b1b3, role.id=4d2118569d074a288b37d120110c300d, scope.project.id=df1f045b58c24b4dba0964eff7d2e540) {{(pid=80119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:18:33.496949 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de1d7e8-1b06-4433-b0e9-199728a1a054 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:18:33.498386 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de1d7e8-1b06-4433-b0e9-199728a1a054 None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:18:33.505092 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 224/450] 10.222.0.49 () {56 vars in 1376 bytes} [Wed Jun 15 10:18:33 2022] GET /identity/v3/role_assignments?group.id=77182ac3056944e19934f133b3f7b1b3&scope.project.id=df1f045b58c24b4dba0964eff7d2e540&role.id=4d2118569d074a288b37d120110c300d => generated 611 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 10:18:33.515607 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880ab57f-ef78-42bd-b724-80562f5ed7ba None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:18:33.562248 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa2adf4-d257-44ae-b614-4bd7ccad2cb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:18:33.562953 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa2adf4-d257-44ae-b614-4bd7ccad2cb2 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:33.563382 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa2adf4-d257-44ae-b614-4bd7ccad2cb2 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:33.563676 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa2adf4-d257-44ae-b614-4bd7ccad2cb2 None admin] PATH_INFO: `/v3/roles` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:33.564463 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa2adf4-d257-44ae-b614-4bd7ccad2cb2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:18:33.564780 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa2adf4-d257-44ae-b614-4bd7ccad2cb2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:18:33.566232 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa2adf4-d257-44ae-b614-4bd7ccad2cb2 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:18:33.576668 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 227/451] 10.222.0.49 () {56 vars in 1099 bytes} [Wed Jun 15 10:18:33 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 10:18:33.582396 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de1d7e8-1b06-4433-b0e9-199728a1a054 None admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:18:33.668385 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=100 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:18:33.689691 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de1d7e8-1b06-4433-b0e9-199728a1a054 None admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:18:33.723716 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78cba496-8421-4a16-920a-fc84c0d4beee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:18:33.724362 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78cba496-8421-4a16-920a-fc84c0d4beee None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:33.724654 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78cba496-8421-4a16-920a-fc84c0d4beee None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:33.724934 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78cba496-8421-4a16-920a-fc84c0d4beee None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:33.797563 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78cba496-8421-4a16-920a-fc84c0d4beee None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:18:33.799044 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78cba496-8421-4a16-920a-fc84c0d4beee None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:18:33.880718 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=100 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:18:33.900401 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 225/452] 10.222.0.49 () {58 vars in 1282 bytes} [Wed Jun 15 10:18:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 15 10:18:33.906000 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa2adf4-d257-44ae-b614-4bd7ccad2cb2 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:18:33.928145 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:18:34.003291 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa2adf4-d257-44ae-b614-4bd7ccad2cb2 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:18:34.034641 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85cf8dd8-4e6c-48d8-bfde-5842b19de609 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:18:34.035521 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85cf8dd8-4e6c-48d8-bfde-5842b19de609 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:34.035807 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85cf8dd8-4e6c-48d8-bfde-5842b19de609 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:34.036083 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85cf8dd8-4e6c-48d8-bfde-5842b19de609 None admin] PATH_INFO: `/v3/projects/c8da8e1de64f469c9cc5f9a5f416060d` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:34.037269 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85cf8dd8-4e6c-48d8-bfde-5842b19de609 None admin] RBAC: Authorizing `identity:get_project(project_id=c8da8e1de64f469c9cc5f9a5f416060d)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:18:34.038695 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85cf8dd8-4e6c-48d8-bfde-5842b19de609 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:18:34.040375 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 228/453] 10.222.0.49 () {56 vars in 1186 bytes} [Wed Jun 15 10:18:33 2022] GET /identity/v3/projects/c8da8e1de64f469c9cc5f9a5f416060d => generated 344 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:18:34.046291 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78cba496-8421-4a16-920a-fc84c0d4beee None admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:18:34.141945 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78cba496-8421-4a16-920a-fc84c0d4beee None admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:18:34.174313 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d22bff-e3d3-49da-a6b4-a1bdcf8d2a68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:18:34.175181 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d22bff-e3d3-49da-a6b4-a1bdcf8d2a68 None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:34.175468 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d22bff-e3d3-49da-a6b4-a1bdcf8d2a68 None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:34.175745 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d22bff-e3d3-49da-a6b4-a1bdcf8d2a68 None admin] PATH_INFO: `/v3/groups/ffc16ae2b8b843bc92af79734c11eeba` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:34.177158 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d22bff-e3d3-49da-a6b4-a1bdcf8d2a68 None admin] RBAC: Authorizing `identity:get_group(group_id=ffc16ae2b8b843bc92af79734c11eeba)` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:18:34.178610 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d22bff-e3d3-49da-a6b4-a1bdcf8d2a68 None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:18:34.180217 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 226/454] 10.222.0.49 () {56 vars in 1180 bytes} [Wed Jun 15 10:18:34 2022] GET /identity/v3/groups/ffc16ae2b8b843bc92af79734c11eeba => generated 229 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:18:34.187257 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85cf8dd8-4e6c-48d8-bfde-5842b19de609 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:18:34.269163 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:18:34.281612 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85cf8dd8-4e6c-48d8-bfde-5842b19de609 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:18:34.313860 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac680bc-7715-4581-ad9e-8a8cb1c45c00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:18:34.314507 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac680bc-7715-4581-ad9e-8a8cb1c45c00 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:34.314830 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac680bc-7715-4581-ad9e-8a8cb1c45c00 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:34.315068 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac680bc-7715-4581-ad9e-8a8cb1c45c00 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:34.315923 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac680bc-7715-4581-ad9e-8a8cb1c45c00 None admin] RBAC: Adding query filter params (group.id=ffc16ae2b8b843bc92af79734c11eeba, role.id=8051174e7a7947d5b991ebd793c2e543, scope.project.id=c8da8e1de64f469c9cc5f9a5f416060d) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:18:34.316235 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac680bc-7715-4581-ad9e-8a8cb1c45c00 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:18:34.317650 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac680bc-7715-4581-ad9e-8a8cb1c45c00 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:18:34.323274 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 229/455] 10.222.0.49 () {56 vars in 1376 bytes} [Wed Jun 15 10:18:34 2022] GET /identity/v3/role_assignments?group.id=ffc16ae2b8b843bc92af79734c11eeba&scope.project.id=c8da8e1de64f469c9cc5f9a5f416060d&role.id=8051174e7a7947d5b991ebd793c2e543 => generated 611 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:18:36.894495 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed85bce8-c0e7-469f-8d1e-a5b92d6e1a40 None None] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:36.894998 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed85bce8-c0e7-469f-8d1e-a5b92d6e1a40 None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:36.895073 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed85bce8-c0e7-469f-8d1e-a5b92d6e1a40 None None] PATH_INFO: `/` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:36.896644 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 227/456] 10.222.0.49 () {54 vars in 926 bytes} [Wed Jun 15 10:18:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:18:36.902242 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead0f1b9-c86a-4436-b9fe-ff65a95b8bc8 None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:36.903251 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead0f1b9-c86a-4436-b9fe-ff65a95b8bc8 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:36.903465 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead0f1b9-c86a-4436-b9fe-ff65a95b8bc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:36.928748 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-ead0f1b9-c86a-4436-b9fe-ff65a95b8bc8 None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:18:36.929952 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-ead0f1b9-c86a-4436-b9fe-ff65a95b8bc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:18:37.024077 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 230/457] 10.222.0.49 () {58 vars in 1023 bytes} [Wed Jun 15 10:18:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 10:18:37.060920 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed85bce8-c0e7-469f-8d1e-a5b92d6e1a40 None None] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:18:37.124465 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=100 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:18:37.156848 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5d098c-bb16-4479-8182-afb54d79b9ab None None] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:37.157143 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5d098c-bb16-4479-8182-afb54d79b9ab None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:37.157402 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5d098c-bb16-4479-8182-afb54d79b9ab None None] PATH_INFO: `/` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:37.159021 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 231/458] 10.222.0.49 () {54 vars in 926 bytes} [Wed Jun 15 10:18:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:18:37.164069 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10284760-aa8d-42a9-8727-bae49d3d40aa None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:37.164353 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10284760-aa8d-42a9-8727-bae49d3d40aa None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:37.164625 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10284760-aa8d-42a9-8727-bae49d3d40aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:37.189302 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed85bce8-c0e7-469f-8d1e-a5b92d6e1a40 None None] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:18:37.207527 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-10284760-aa8d-42a9-8727-bae49d3d40aa None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:18:37.209348 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-10284760-aa8d-42a9-8727-bae49d3d40aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:18:37.235134 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4066bf22-eebb-4e7f-b521-81513a04e8df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:18:37.235134 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4066bf22-eebb-4e7f-b521-81513a04e8df None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:37.235134 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4066bf22-eebb-4e7f-b521-81513a04e8df None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:37.235134 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4066bf22-eebb-4e7f-b521-81513a04e8df None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:37.247342 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4066bf22-eebb-4e7f-b521-81513a04e8df None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:18:37.248826 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4066bf22-eebb-4e7f-b521-81513a04e8df None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:18:37.279608 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application [None req-4066bf22-eebb-4e7f-b521-81513a04e8df None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 10:18:37.279608 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:18:37.279608 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:18:37.279608 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:18:37.279608 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:18:37.279608 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:18:37.279608 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:18:37.279608 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:18:37.279608 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:18:37.279608 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:18:37.279608 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:18:37.279608 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:18:37.279608 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 10:18:37.279608 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 10:18:37.279608 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 10:18:37.280533 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 10:18:37.280533 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:18:37.280533 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:18:37.280533 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:18:37.280533 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:18:37.280533 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:18:37.280533 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:18:37.280533 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:18:37.280533 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application with Lock( Jun 15 10:18:37.280533 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:18:37.280533 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:18:37.280533 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:18:37.280533 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:18:37.280533 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:18:37.280533 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:18:37.280533 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:18:37.280533 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:18:37.281374 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 10:18:37.281374 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 10:18:37.281374 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 10:18:37.281374 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application Jun 15 10:18:37.281374 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 228/459] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Jun 15 10:18:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:18:37.286351 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee0e296-11a0-412b-ab50-6bd4d79772fc None None] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:37.286351 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee0e296-11a0-412b-ab50-6bd4d79772fc None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:37.286351 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee0e296-11a0-412b-ab50-6bd4d79772fc None None] PATH_INFO: `/` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:37.286351 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 229/460] 10.222.0.49 () {54 vars in 926 bytes} [Wed Jun 15 10:18:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:18:37.287047 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee0e296-11a0-412b-ab50-6bd4d79772fc None None] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:18:37.324996 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 232/461] 10.222.0.49 () {58 vars in 1023 bytes} [Wed Jun 15 10:18:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 10:18:37.327404 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368b2256-2386-4207-b780-f84975dbbe6d None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:37.327704 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368b2256-2386-4207-b780-f84975dbbe6d None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:37.327986 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368b2256-2386-4207-b780-f84975dbbe6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:37.363775 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-368b2256-2386-4207-b780-f84975dbbe6d None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:18:37.365025 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-368b2256-2386-4207-b780-f84975dbbe6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:18:37.383099 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:18:37.406775 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=100 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:18:37.411566 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee0e296-11a0-412b-ab50-6bd4d79772fc None None] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:18:37.448290 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d59abf-8dcb-441c-95af-3c1aab5769f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:18:37.449112 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d59abf-8dcb-441c-95af-3c1aab5769f4 None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:37.450164 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d59abf-8dcb-441c-95af-3c1aab5769f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:37.450538 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d59abf-8dcb-441c-95af-3c1aab5769f4 None admin] PATH_INFO: `/v3/domains` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:37.451431 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d59abf-8dcb-441c-95af-3c1aab5769f4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:18:37.451935 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d59abf-8dcb-441c-95af-3c1aab5769f4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:18:37.453559 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d59abf-8dcb-441c-95af-3c1aab5769f4 None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:18:37.453739 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 233/462] 10.222.0.49 () {58 vars in 1023 bytes} [Wed Jun 15 10:18:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 10:18:37.454768 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368b2256-2386-4207-b780-f84975dbbe6d None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:18:37.493798 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 230/463] 10.222.0.49 () {56 vars in 1109 bytes} [Wed Jun 15 10:18:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 10:18:37.499781 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d59abf-8dcb-441c-95af-3c1aab5769f4 None admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:18:37.572187 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368b2256-2386-4207-b780-f84975dbbe6d None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:18:37.607790 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a35db1-a1eb-498e-b9e3-ce8dbf173299 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:18:37.607955 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d59abf-8dcb-441c-95af-3c1aab5769f4 None admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:18:37.608523 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a35db1-a1eb-498e-b9e3-ce8dbf173299 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:37.608807 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a35db1-a1eb-498e-b9e3-ce8dbf173299 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:37.609082 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a35db1-a1eb-498e-b9e3-ce8dbf173299 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:37.619529 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a35db1-a1eb-498e-b9e3-ce8dbf173299 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:18:37.620982 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a35db1-a1eb-498e-b9e3-ce8dbf173299 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:18:37.623539 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:18:37.641342 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application [None req-23a35db1-a1eb-498e-b9e3-ce8dbf173299 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 10:18:37.641342 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:18:37.641342 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:18:37.641342 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:18:37.641342 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:18:37.641342 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:18:37.641342 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:18:37.641342 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:18:37.641342 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:18:37.641342 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:18:37.641342 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:18:37.641342 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:18:37.641342 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 10:18:37.641342 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 10:18:37.641342 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 10:18:37.642226 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 10:18:37.642226 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:18:37.642226 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:18:37.642226 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:18:37.642226 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:18:37.642226 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:18:37.642226 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:18:37.642226 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:18:37.642226 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application with Lock( Jun 15 10:18:37.642226 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:18:37.642226 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:18:37.642226 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:18:37.642226 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:18:37.642226 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:18:37.642226 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:18:37.642226 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:18:37.642226 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:18:37.643081 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 10:18:37.643081 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 10:18:37.643081 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 10:18:37.643081 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Jun 15 10:18:37.643081 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 234/464] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Jun 15 10:18:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:18:37.643376 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a35db1-a1eb-498e-b9e3-ce8dbf173299 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:18:37.648604 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10e42ce-5126-4d9c-9a52-373c538db7c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:18:37.649355 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10e42ce-5126-4d9c-9a52-373c538db7c0 None admin] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:37.649619 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10e42ce-5126-4d9c-9a52-373c538db7c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:37.649895 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10e42ce-5126-4d9c-9a52-373c538db7c0 None admin] PATH_INFO: `/v3/users` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:37.650596 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d10e42ce-5126-4d9c-9a52-373c538db7c0 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:18:37.652089 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d10e42ce-5126-4d9c-9a52-373c538db7c0 None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:18:37.674321 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 231/465] 10.222.0.49 () {60 vars in 1132 bytes} [Wed Jun 15 10:18:37 2022] POST /identity/v3/users => generated 246 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 10:18:37.675623 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28a882c-89e3-4b77-a52e-c5d938c4df6e None None] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:37.675967 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28a882c-89e3-4b77-a52e-c5d938c4df6e None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:37.676511 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28a882c-89e3-4b77-a52e-c5d938c4df6e None None] PATH_INFO: `/` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:37.677914 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 232/466] 10.222.0.49 () {54 vars in 926 bytes} [Wed Jun 15 10:18:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:18:37.682681 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28a882c-89e3-4b77-a52e-c5d938c4df6e None None] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:18:37.719744 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:18:37.776812 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a35db1-a1eb-498e-b9e3-ce8dbf173299 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:18:37.823450 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92bbc010-4265-4bf7-b6b2-fb714dfd52db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:18:37.824374 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bbc010-4265-4bf7-b6b2-fb714dfd52db None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:37.824717 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bbc010-4265-4bf7-b6b2-fb714dfd52db None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:37.825472 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bbc010-4265-4bf7-b6b2-fb714dfd52db None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:37.841916 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92bbc010-4265-4bf7-b6b2-fb714dfd52db None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:18:37.843442 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92bbc010-4265-4bf7-b6b2-fb714dfd52db None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:18:37.850937 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28a882c-89e3-4b77-a52e-c5d938c4df6e None None] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:18:37.868596 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application [None req-92bbc010-4265-4bf7-b6b2-fb714dfd52db None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 10:18:37.868596 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:18:37.868596 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:18:37.868596 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:18:37.868596 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:18:37.868596 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:18:37.868596 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:18:37.868596 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:18:37.868596 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:18:37.868596 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:18:37.868596 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:18:37.868596 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:18:37.868596 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 10:18:37.868596 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 10:18:37.868596 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 10:18:37.869463 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 10:18:37.869463 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:18:37.869463 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:18:37.869463 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:18:37.869463 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:18:37.869463 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:18:37.869463 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:18:37.869463 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:18:37.869463 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application with Lock( Jun 15 10:18:37.869463 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:18:37.869463 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:18:37.869463 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:18:37.869463 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:18:37.869463 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:18:37.869463 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:18:37.869463 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:18:37.869463 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:18:37.870318 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 10:18:37.870318 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 10:18:37.870318 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 10:18:37.870318 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Jun 15 10:18:37.870927 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 235/467] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Jun 15 10:18:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:18:37.872823 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2800ca-713b-4293-a296-0cd82efa0e2f None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:37.873111 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2800ca-713b-4293-a296-0cd82efa0e2f None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:37.873389 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2800ca-713b-4293-a296-0cd82efa0e2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:37.892182 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-fb2800ca-713b-4293-a296-0cd82efa0e2f None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:18:37.893435 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-fb2800ca-713b-4293-a296-0cd82efa0e2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:18:37.894008 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909a85ca-b86c-476b-8a71-80ea51302613 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:18:37.894677 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909a85ca-b86c-476b-8a71-80ea51302613 None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:37.895225 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909a85ca-b86c-476b-8a71-80ea51302613 None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:37.895298 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909a85ca-b86c-476b-8a71-80ea51302613 None admin] PATH_INFO: `/v3/domains` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:37.895973 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909a85ca-b86c-476b-8a71-80ea51302613 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:18:37.896280 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909a85ca-b86c-476b-8a71-80ea51302613 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:18:37.897711 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909a85ca-b86c-476b-8a71-80ea51302613 None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:18:37.911903 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 233/468] 10.222.0.49 () {56 vars in 1109 bytes} [Wed Jun 15 10:18:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:18:37.912945 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909a85ca-b86c-476b-8a71-80ea51302613 None admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:18:37.941779 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:18:37.979646 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 236/469] 10.222.0.49 () {58 vars in 1023 bytes} [Wed Jun 15 10:18:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 10:18:37.980948 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2800ca-713b-4293-a296-0cd82efa0e2f None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:18:38.009937 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=100 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:18:38.044854 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909a85ca-b86c-476b-8a71-80ea51302613 None admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:18:38.082688 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a253015-e977-4814-b4c2-675aa426df93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:18:38.083391 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a253015-e977-4814-b4c2-675aa426df93 None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:38.083675 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a253015-e977-4814-b4c2-675aa426df93 None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:38.083951 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a253015-e977-4814-b4c2-675aa426df93 None admin] PATH_INFO: `/v3/domains` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:38.084364 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2800ca-713b-4293-a296-0cd82efa0e2f None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:18:38.084674 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a253015-e977-4814-b4c2-675aa426df93 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:18:38.084983 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a253015-e977-4814-b4c2-675aa426df93 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:18:38.086492 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a253015-e977-4814-b4c2-675aa426df93 None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:18:38.099893 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 234/470] 10.222.0.49 () {56 vars in 1109 bytes} [Wed Jun 15 10:18:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:18:38.100584 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a253015-e977-4814-b4c2-675aa426df93 None admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:18:38.120915 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00916ff7-8ada-4484-977c-9939f1e6aaed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:18:38.121706 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00916ff7-8ada-4484-977c-9939f1e6aaed None admin] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:38.121995 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00916ff7-8ada-4484-977c-9939f1e6aaed None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:38.122572 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00916ff7-8ada-4484-977c-9939f1e6aaed None admin] PATH_INFO: `/v3/users` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:38.123311 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00916ff7-8ada-4484-977c-9939f1e6aaed None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:18:38.124814 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00916ff7-8ada-4484-977c-9939f1e6aaed None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:18:38.158299 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 237/471] 10.222.0.49 () {60 vars in 1132 bytes} [Wed Jun 15 10:18:37 2022] POST /identity/v3/users => generated 248 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 10:18:38.159286 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00916ff7-8ada-4484-977c-9939f1e6aaed None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:18:38.208801 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:18:38.215664 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a253015-e977-4814-b4c2-675aa426df93 None admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:18:38.256949 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee06ab0-aca2-4827-90c6-bdf229d795c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:18:38.257854 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee06ab0-aca2-4827-90c6-bdf229d795c8 None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:38.258211 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee06ab0-aca2-4827-90c6-bdf229d795c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:38.258555 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee06ab0-aca2-4827-90c6-bdf229d795c8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:38.262850 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=100 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:18:38.274904 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee06ab0-aca2-4827-90c6-bdf229d795c8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:18:38.277866 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee06ab0-aca2-4827-90c6-bdf229d795c8 None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:18:38.302629 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application [None req-7ee06ab0-aca2-4827-90c6-bdf229d795c8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 10:18:38.302629 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:18:38.302629 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:18:38.302629 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:18:38.302629 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:18:38.302629 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:18:38.302629 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:18:38.302629 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:18:38.302629 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:18:38.302629 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:18:38.302629 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:18:38.302629 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:18:38.302629 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 10:18:38.302629 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 10:18:38.302629 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 10:18:38.304413 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 10:18:38.304413 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:18:38.304413 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:18:38.304413 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:18:38.304413 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:18:38.304413 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:18:38.304413 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:18:38.304413 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:18:38.304413 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application with Lock( Jun 15 10:18:38.304413 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:18:38.304413 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:18:38.304413 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:18:38.304413 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:18:38.304413 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:18:38.304413 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:18:38.304413 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:18:38.304413 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:18:38.306392 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 10:18:38.306392 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 10:18:38.306392 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 10:18:38.306392 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application Jun 15 10:18:38.306877 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00916ff7-8ada-4484-977c-9939f1e6aaed None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:18:38.307020 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 235/472] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Jun 15 10:18:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:18:38.316686 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee06ab0-aca2-4827-90c6-bdf229d795c8 None admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:18:38.392062 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0cdc3c-e4a2-459b-a133-3da68c6b7d79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:18:38.392687 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0cdc3c-e4a2-459b-a133-3da68c6b7d79 None admin] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:38.392977 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0cdc3c-e4a2-459b-a133-3da68c6b7d79 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:38.393255 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0cdc3c-e4a2-459b-a133-3da68c6b7d79 None admin] PATH_INFO: `/v3/users` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:38.393988 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be0cdc3c-e4a2-459b-a133-3da68c6b7d79 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:18:38.395488 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be0cdc3c-e4a2-459b-a133-3da68c6b7d79 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:18:38.434727 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 238/473] 10.222.0.49 () {60 vars in 1132 bytes} [Wed Jun 15 10:18:38 2022] POST /identity/v3/users => generated 248 bytes in 276 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 10:18:38.460831 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9b0c0a-14f1-4d2c-a2ac-90680de5e80e None None] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:38.461259 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9b0c0a-14f1-4d2c-a2ac-90680de5e80e None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:38.461638 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9b0c0a-14f1-4d2c-a2ac-90680de5e80e None None] PATH_INFO: `/` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:38.463372 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 239/474] 10.222.0.49 () {54 vars in 926 bytes} [Wed Jun 15 10:18:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:18:38.468873 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b99cd4-7845-40bb-92c5-f70d65085b3b None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:38.469257 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b99cd4-7845-40bb-92c5-f70d65085b3b None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:38.469632 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b99cd4-7845-40bb-92c5-f70d65085b3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:38.485330 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee06ab0-aca2-4827-90c6-bdf229d795c8 None admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:18:38.489264 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-39b99cd4-7845-40bb-92c5-f70d65085b3b None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:18:38.489884 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-39b99cd4-7845-40bb-92c5-f70d65085b3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:18:38.528279 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a017bad-cf39-4f57-beca-1004dc4a6d38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:18:38.529218 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a017bad-cf39-4f57-beca-1004dc4a6d38 None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:38.529650 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a017bad-cf39-4f57-beca-1004dc4a6d38 None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:38.530099 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a017bad-cf39-4f57-beca-1004dc4a6d38 None admin] PATH_INFO: `/v3/domains` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:38.531058 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a017bad-cf39-4f57-beca-1004dc4a6d38 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:18:38.531542 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a017bad-cf39-4f57-beca-1004dc4a6d38 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:18:38.533324 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a017bad-cf39-4f57-beca-1004dc4a6d38 None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:18:38.548409 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 236/475] 10.222.0.49 () {56 vars in 1109 bytes} [Wed Jun 15 10:18:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 10:18:38.554465 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a017bad-cf39-4f57-beca-1004dc4a6d38 None admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:18:38.570029 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 240/476] 10.222.0.49 () {58 vars in 1023 bytes} [Wed Jun 15 10:18:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 10:18:38.593824 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b99cd4-7845-40bb-92c5-f70d65085b3b None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:18:38.616533 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=100 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:18:38.634362 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:18:38.693416 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a017bad-cf39-4f57-beca-1004dc4a6d38 None admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:18:38.728454 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b99cd4-7845-40bb-92c5-f70d65085b3b None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:18:38.729663 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eaaea2a-50e4-47d0-81b3-66b91ed85819 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:18:38.730966 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eaaea2a-50e4-47d0-81b3-66b91ed85819 None admin] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:38.731344 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eaaea2a-50e4-47d0-81b3-66b91ed85819 None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:38.731755 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eaaea2a-50e4-47d0-81b3-66b91ed85819 None admin] PATH_INFO: `/v3/users` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:38.732468 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eaaea2a-50e4-47d0-81b3-66b91ed85819 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:18:38.733921 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eaaea2a-50e4-47d0-81b3-66b91ed85819 None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:18:38.756711 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 237/477] 10.222.0.49 () {60 vars in 1132 bytes} [Wed Jun 15 10:18:38 2022] POST /identity/v3/users => generated 249 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 10:18:38.778686 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596a93b8-67c8-4df7-9683-6cb5b389b0d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:18:38.779551 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596a93b8-67c8-4df7-9683-6cb5b389b0d1 None admin] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:38.779867 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596a93b8-67c8-4df7-9683-6cb5b389b0d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:38.780156 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596a93b8-67c8-4df7-9683-6cb5b389b0d1 None admin] PATH_INFO: `/v3/roles` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:38.781036 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-596a93b8-67c8-4df7-9683-6cb5b389b0d1 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:18:38.782655 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-596a93b8-67c8-4df7-9683-6cb5b389b0d1 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:18:38.794925 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.sql.core [None req-596a93b8-67c8-4df7-9683-6cb5b389b0d1 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jun 15 10:18:38.794925 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jun 15 10:18:38.794925 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [parameters: {'id': '84b3c64cf4ba468d9f3bd9a329b1fb14', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jun 15 10:18:38.794925 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80120) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jun 15 10:18:38.812857 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application [None req-596a93b8-67c8-4df7-9683-6cb5b389b0d1 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jun 15 10:18:38.812857 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:18:38.812857 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 15 10:18:38.812857 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 15 10:18:38.812857 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 15 10:18:38.812857 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 15 10:18:38.812857 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 15 10:18:38.812857 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application result = self._query(query) Jun 15 10:18:38.812857 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 15 10:18:38.812857 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application conn.query(q) Jun 15 10:18:38.812857 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 15 10:18:38.812857 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 15 10:18:38.812857 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 15 10:18:38.812857 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application result.read() Jun 15 10:18:38.812857 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 15 10:18:38.813816 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 15 10:18:38.813816 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 15 10:18:38.813816 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 15 10:18:38.813816 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 15 10:18:38.813816 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 15 10:18:38.813816 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 15 10:18:38.813816 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 15 10:18:38.813816 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jun 15 10:18:38.813816 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Jun 15 10:18:38.813816 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jun 15 10:18:38.813816 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Jun 15 10:18:38.813816 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:18:38.813816 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jun 15 10:18:38.813816 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jun 15 10:18:38.813816 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Jun 15 10:18:38.813816 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return ref.to_dict() Jun 15 10:18:38.813816 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 15 10:18:38.813816 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application next(self.gen) Jun 15 10:18:38.814807 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jun 15 10:18:38.814807 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application yield resource Jun 15 10:18:38.814807 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 15 10:18:38.814807 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application next(self.gen) Jun 15 10:18:38.814807 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jun 15 10:18:38.814807 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application self.session.rollback() Jun 15 10:18:38.814807 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jun 15 10:18:38.814807 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application self.force_reraise() Jun 15 10:18:38.814807 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jun 15 10:18:38.814807 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application raise self.value Jun 15 10:18:38.814807 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jun 15 10:18:38.814807 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jun 15 10:18:38.814807 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jun 15 10:18:38.814807 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application session.commit() Jun 15 10:18:38.814807 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jun 15 10:18:38.814807 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jun 15 10:18:38.814807 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jun 15 10:18:38.815701 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application self._prepare_impl() Jun 15 10:18:38.815701 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jun 15 10:18:38.815701 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application self.session.flush() Jun 15 10:18:38.815701 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jun 15 10:18:38.815701 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application self._flush(objects) Jun 15 10:18:38.815701 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jun 15 10:18:38.815701 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jun 15 10:18:38.815701 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jun 15 10:18:38.815701 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application compat.raise_( Jun 15 10:18:38.815701 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 15 10:18:38.815701 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application raise exception Jun 15 10:18:38.815701 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jun 15 10:18:38.815701 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application flush_context.execute() Jun 15 10:18:38.815701 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jun 15 10:18:38.815701 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application rec.execute(self) Jun 15 10:18:38.815701 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jun 15 10:18:38.815701 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jun 15 10:18:38.816596 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jun 15 10:18:38.816596 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application _emit_insert_statements( Jun 15 10:18:38.816596 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jun 15 10:18:38.816596 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application c = connection._execute_20( Jun 15 10:18:38.816596 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jun 15 10:18:38.816596 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jun 15 10:18:38.816596 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection Jun 15 10:18:38.816596 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jun 15 10:18:38.816596 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jun 15 10:18:38.816596 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application ret = self._execute_context( Jun 15 10:18:38.816596 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jun 15 10:18:38.816596 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jun 15 10:18:38.816596 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jun 15 10:18:38.816596 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jun 15 10:18:38.816596 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 15 10:18:38.816596 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application raise exception Jun 15 10:18:38.817421 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 15 10:18:38.817421 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 15 10:18:38.817421 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 15 10:18:38.817421 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 15 10:18:38.817421 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 15 10:18:38.817421 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application result = self._query(query) Jun 15 10:18:38.817421 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 15 10:18:38.817421 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application conn.query(q) Jun 15 10:18:38.817421 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 15 10:18:38.817421 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 15 10:18:38.817421 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 15 10:18:38.817421 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application result.read() Jun 15 10:18:38.817421 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 15 10:18:38.817421 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 15 10:18:38.817421 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 15 10:18:38.817421 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 15 10:18:38.818225 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 15 10:18:38.818225 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 15 10:18:38.818225 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 15 10:18:38.818225 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 15 10:18:38.818225 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jun 15 10:18:38.818225 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jun 15 10:18:38.818225 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application [parameters: {'id': '84b3c64cf4ba468d9f3bd9a329b1fb14', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jun 15 10:18:38.818225 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jun 15 10:18:38.818225 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Jun 15 10:18:38.818225 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 15 10:18:38.818225 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Jun 15 10:18:38.818225 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:18:38.818225 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:18:38.818225 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:18:38.818225 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:18:38.818225 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:18:38.820574 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:18:38.820574 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:18:38.820574 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:18:38.820574 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:18:38.820574 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:18:38.820574 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:18:38.820574 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Jun 15 10:18:38.820574 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Jun 15 10:18:38.820574 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:18:38.820574 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:18:38.820574 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Jun 15 10:18:38.820574 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Jun 15 10:18:38.820574 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jun 15 10:18:38.820574 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jun 15 10:18:38.820574 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jun 15 10:18:38.820574 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Jun 15 10:18:38.822460 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 241/478] 10.222.0.49 () {60 vars in 1131 bytes} [Wed Jun 15 10:18:38 2022] POST /identity/v3/roles => generated 144 bytes in 222 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:18:38.822546 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eaaea2a-50e4-47d0-81b3-66b91ed85819 None admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:18:38.948369 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eaaea2a-50e4-47d0-81b3-66b91ed85819 None admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:18:38.950822 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:18:38.989402 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082bd76d-0224-40f9-a33d-764ce0a81d86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:18:38.990240 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082bd76d-0224-40f9-a33d-764ce0a81d86 None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:38.990523 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082bd76d-0224-40f9-a33d-764ce0a81d86 None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:38.990795 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082bd76d-0224-40f9-a33d-764ce0a81d86 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:39.002448 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-082bd76d-0224-40f9-a33d-764ce0a81d86 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 10:18:39.010577 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application [None req-082bd76d-0224-40f9-a33d-764ce0a81d86 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 15 10:18:39.010577 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:18:39.010577 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:18:39.010577 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:18:39.010577 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:18:39.010577 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:18:39.010577 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:18:39.010577 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:18:39.010577 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:18:39.010577 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:18:39.010577 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:18:39.010577 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:18:39.010577 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 10:18:39.010577 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 10:18:39.010577 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 10:18:39.010577 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 10:18:39.011523 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 10:18:39.011523 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 10:18:39.011523 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 10:18:39.011523 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 10:18:39.011523 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:18:39.011523 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:18:39.011523 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:18:39.011523 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:18:39.011523 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:18:39.011523 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:18:39.011523 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:18:39.011523 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application with Lock( Jun 15 10:18:39.011523 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:18:39.011523 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:18:39.011523 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:18:39.011523 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:18:39.012378 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:18:39.012378 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:18:39.012378 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:18:39.012378 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:18:39.012378 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 10:18:39.012378 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 10:18:39.012378 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 10:18:39.012378 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 10:18:39.012378 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 10:18:39.012378 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 10:18:39.012378 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 15 10:18:39.012378 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application Jun 15 10:18:39.012378 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 238/479] 10.222.0.49 () {56 vars in 1114 bytes} [Wed Jun 15 10:18:38 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:18:39.018124 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596a93b8-67c8-4df7-9683-6cb5b389b0d1 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:18:39.123241 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:18:39.146326 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596a93b8-67c8-4df7-9683-6cb5b389b0d1 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:18:39.182102 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12c04e1-8e1f-4aaf-a4a1-c5041058bc59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:18:39.182786 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12c04e1-8e1f-4aaf-a4a1-c5041058bc59 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:39.183073 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12c04e1-8e1f-4aaf-a4a1-c5041058bc59 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:39.183373 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12c04e1-8e1f-4aaf-a4a1-c5041058bc59 None admin] PATH_INFO: `/v3/roles` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:39.184168 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f12c04e1-8e1f-4aaf-a4a1-c5041058bc59 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:18:39.184476 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f12c04e1-8e1f-4aaf-a4a1-c5041058bc59 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:18:39.185860 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f12c04e1-8e1f-4aaf-a4a1-c5041058bc59 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:18:39.194710 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 242/480] 10.222.0.49 () {56 vars in 1111 bytes} [Wed Jun 15 10:18:39 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:18:40.324187 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b66ddd-3fa5-48fa-8e7f-f46e67842bb1 None None] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:40.325197 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b66ddd-3fa5-48fa-8e7f-f46e67842bb1 None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:40.325706 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b66ddd-3fa5-48fa-8e7f-f46e67842bb1 None None] PATH_INFO: `/` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:40.327556 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 239/481] 10.222.0.49 () {54 vars in 926 bytes} [Wed Jun 15 10:18:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:18:40.332638 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f4e578-ebc2-43e5-aed6-ddff5364480c None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:40.333038 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f4e578-ebc2-43e5-aed6-ddff5364480c None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:40.333204 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f4e578-ebc2-43e5-aed6-ddff5364480c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:40.354973 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-f2f4e578-ebc2-43e5-aed6-ddff5364480c None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:18:40.356761 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-f2f4e578-ebc2-43e5-aed6-ddff5364480c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:18:40.438081 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 243/482] 10.222.0.49 () {58 vars in 1023 bytes} [Wed Jun 15 10:18:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 10:18:40.481941 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b66ddd-3fa5-48fa-8e7f-f46e67842bb1 None None] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:18:40.615541 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b66ddd-3fa5-48fa-8e7f-f46e67842bb1 None None] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:18:40.651607 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b0371e-e6dd-4309-ab4a-7f14f0ec98a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:18:40.652466 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b0371e-e6dd-4309-ab4a-7f14f0ec98a7 None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:40.652757 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b0371e-e6dd-4309-ab4a-7f14f0ec98a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:40.653049 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b0371e-e6dd-4309-ab4a-7f14f0ec98a7 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:40.663806 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b0371e-e6dd-4309-ab4a-7f14f0ec98a7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 10:18:40.672278 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application [None req-25b0371e-e6dd-4309-ab4a-7f14f0ec98a7 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 15 10:18:40.672278 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:18:40.672278 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:18:40.672278 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:18:40.672278 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:18:40.672278 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:18:40.672278 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:18:40.672278 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:18:40.672278 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:18:40.672278 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:18:40.672278 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:18:40.672278 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:18:40.672278 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 10:18:40.672278 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 10:18:40.672278 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 10:18:40.672278 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 10:18:40.673235 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 10:18:40.673235 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 10:18:40.673235 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 10:18:40.673235 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 10:18:40.673235 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:18:40.673235 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:18:40.673235 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:18:40.673235 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:18:40.673235 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:18:40.673235 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:18:40.673235 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:18:40.673235 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application with Lock( Jun 15 10:18:40.673235 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:18:40.673235 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:18:40.673235 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:18:40.673235 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:18:40.674023 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:18:40.674023 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:18:40.674023 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:18:40.674023 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:18:40.674023 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 10:18:40.674023 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 10:18:40.674023 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 10:18:40.674023 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 10:18:40.674023 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 10:18:40.674023 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 10:18:40.674023 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 15 10:18:40.674023 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application Jun 15 10:18:40.674023 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 240/483] 10.222.0.49 () {56 vars in 1102 bytes} [Wed Jun 15 10:18:40 2022] GET /identity/v3/roles/service => generated 85 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:18:40.680060 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f4e578-ebc2-43e5-aed6-ddff5364480c None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:18:40.765135 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925e8d74-9c97-4735-869e-33cd58fb0858 None None] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:40.765549 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925e8d74-9c97-4735-869e-33cd58fb0858 None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:40.765844 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925e8d74-9c97-4735-869e-33cd58fb0858 None None] PATH_INFO: `/` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:40.767569 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 241/484] 10.222.0.49 () {54 vars in 926 bytes} [Wed Jun 15 10:18:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:18:40.775462 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46b757e-e685-4991-9294-76c5f190a556 None None] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:40.775761 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46b757e-e685-4991-9294-76c5f190a556 None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:40.776037 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46b757e-e685-4991-9294-76c5f190a556 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:40.803242 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.auth.core [None req-e46b757e-e685-4991-9294-76c5f190a556 None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:18:40.804480 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-e46b757e-e685-4991-9294-76c5f190a556 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:18:40.835571 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:18:40.841950 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=100 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:18:40.879398 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f4e578-ebc2-43e5-aed6-ddff5364480c None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:18:40.931660 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 242/485] 10.222.0.49 () {58 vars in 1023 bytes} [Wed Jun 15 10:18:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 15 10:18:40.939347 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162345e4-00a0-4668-a1ae-cccb67ea341d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:18:40.940021 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162345e4-00a0-4668-a1ae-cccb67ea341d None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:40.940306 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162345e4-00a0-4668-a1ae-cccb67ea341d None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:40.940584 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162345e4-00a0-4668-a1ae-cccb67ea341d None admin] PATH_INFO: `/v3/roles` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:40.941330 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-162345e4-00a0-4668-a1ae-cccb67ea341d None admin] RBAC: Adding query filter params (name=service) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:18:40.941640 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-162345e4-00a0-4668-a1ae-cccb67ea341d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:18:40.943017 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-162345e4-00a0-4668-a1ae-cccb67ea341d None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:18:40.955864 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 244/486] 10.222.0.49 () {56 vars in 1103 bytes} [Wed Jun 15 10:18:40 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:18:40.957691 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46b757e-e685-4991-9294-76c5f190a556 None None] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:18:40.971420 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162345e4-00a0-4668-a1ae-cccb67ea341d None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:18:41.119431 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46b757e-e685-4991-9294-76c5f190a556 None None] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:18:41.145582 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162345e4-00a0-4668-a1ae-cccb67ea341d None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:18:41.161825 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e748835-f8e9-407c-882f-b9a2a8a399ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:18:41.162659 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e748835-f8e9-407c-882f-b9a2a8a399ab None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:41.162946 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e748835-f8e9-407c-882f-b9a2a8a399ab None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:41.163253 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e748835-f8e9-407c-882f-b9a2a8a399ab None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:41.182229 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e748835-f8e9-407c-882f-b9a2a8a399ab None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:18:41.183851 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e748835-f8e9-407c-882f-b9a2a8a399ab None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:18:41.212921 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application [None req-6e748835-f8e9-407c-882f-b9a2a8a399ab None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 10:18:41.212921 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:18:41.212921 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:18:41.212921 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:18:41.212921 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:18:41.212921 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:18:41.212921 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:18:41.212921 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:18:41.212921 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:18:41.212921 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:18:41.212921 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:18:41.212921 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:18:41.212921 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 10:18:41.212921 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 10:18:41.212921 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 10:18:41.213827 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 10:18:41.213827 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:18:41.213827 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:18:41.213827 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:18:41.213827 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:18:41.213827 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:18:41.213827 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:18:41.213827 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:18:41.213827 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application with Lock( Jun 15 10:18:41.213827 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:18:41.213827 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:18:41.213827 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:18:41.213827 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:18:41.213827 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:18:41.213827 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:18:41.213827 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:18:41.213827 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:18:41.215226 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 10:18:41.215226 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 10:18:41.215226 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 10:18:41.215226 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application Jun 15 10:18:41.215226 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 243/487] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Jun 15 10:18:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:18:41.224009 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e748835-f8e9-407c-882f-b9a2a8a399ab None admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:18:41.241139 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01dac4e9-f7a6-4500-9702-dd5efd6097a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:18:41.241781 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=100 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:18:41.241979 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01dac4e9-f7a6-4500-9702-dd5efd6097a9 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:41.242259 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01dac4e9-f7a6-4500-9702-dd5efd6097a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:41.242532 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01dac4e9-f7a6-4500-9702-dd5efd6097a9 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:41.250851 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:18:41.260483 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01dac4e9-f7a6-4500-9702-dd5efd6097a9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 10:18:41.268277 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application [None req-01dac4e9-f7a6-4500-9702-dd5efd6097a9 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 15 10:18:41.268277 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:18:41.268277 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:18:41.268277 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:18:41.268277 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:18:41.268277 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:18:41.268277 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:18:41.268277 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:18:41.268277 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:18:41.268277 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:18:41.268277 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:18:41.268277 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:18:41.268277 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 10:18:41.268277 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 10:18:41.268277 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 10:18:41.268277 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 10:18:41.269256 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 10:18:41.269256 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 10:18:41.269256 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 10:18:41.269256 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 10:18:41.269256 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:18:41.269256 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:18:41.269256 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:18:41.269256 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:18:41.269256 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:18:41.269256 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:18:41.269256 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:18:41.269256 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application with Lock( Jun 15 10:18:41.269256 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:18:41.269256 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:18:41.269256 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:18:41.269256 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:18:41.270051 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:18:41.270051 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:18:41.270051 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:18:41.270051 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:18:41.270051 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 10:18:41.270051 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 10:18:41.270051 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 10:18:41.270051 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 10:18:41.270051 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 10:18:41.270051 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 10:18:41.270051 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 15 10:18:41.270051 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Jun 15 10:18:41.270051 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 245/488] 10.222.0.49 () {56 vars in 1102 bytes} [Wed Jun 15 10:18:40 2022] GET /identity/v3/roles/service => generated 85 bytes in 299 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:18:41.271069 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6310afc4-8bd9-419a-8f5b-3f9da5b4d4af None None] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:41.271507 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6310afc4-8bd9-419a-8f5b-3f9da5b4d4af None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:41.272084 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6310afc4-8bd9-419a-8f5b-3f9da5b4d4af None None] PATH_INFO: `/` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:41.277037 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 246/489] 10.222.0.49 () {54 vars in 926 bytes} [Wed Jun 15 10:18:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:18:41.278334 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6310afc4-8bd9-419a-8f5b-3f9da5b4d4af None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:18:41.337012 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e748835-f8e9-407c-882f-b9a2a8a399ab None admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:18:41.374774 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc926077-3eda-4745-a09c-408a0a62ec59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:18:41.375771 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc926077-3eda-4745-a09c-408a0a62ec59 None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:41.376203 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc926077-3eda-4745-a09c-408a0a62ec59 None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:41.376628 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc926077-3eda-4745-a09c-408a0a62ec59 None admin] PATH_INFO: `/v3/domains` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:41.377502 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc926077-3eda-4745-a09c-408a0a62ec59 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:18:41.378061 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc926077-3eda-4745-a09c-408a0a62ec59 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:18:41.379705 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc926077-3eda-4745-a09c-408a0a62ec59 None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:18:41.392168 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6310afc4-8bd9-419a-8f5b-3f9da5b4d4af None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:18:41.393501 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 244/490] 10.222.0.49 () {56 vars in 1109 bytes} [Wed Jun 15 10:18:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:18:41.395017 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4af7961-3432-4da5-ae4f-2f7c595d688e None None] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:41.395534 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4af7961-3432-4da5-ae4f-2f7c595d688e None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:41.396090 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4af7961-3432-4da5-ae4f-2f7c595d688e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:41.414344 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.auth.core [None req-e4af7961-3432-4da5-ae4f-2f7c595d688e None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:18:41.416064 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-e4af7961-3432-4da5-ae4f-2f7c595d688e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:18:41.433939 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a8aefd-1694-4c8a-8d46-022125ba624f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:18:41.434668 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a8aefd-1694-4c8a-8d46-022125ba624f None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:41.435176 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a8aefd-1694-4c8a-8d46-022125ba624f None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:41.435510 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a8aefd-1694-4c8a-8d46-022125ba624f None admin] PATH_INFO: `/v3/roles` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:41.436239 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a8aefd-1694-4c8a-8d46-022125ba624f None admin] RBAC: Adding query filter params (name=service) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:18:41.436549 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a8aefd-1694-4c8a-8d46-022125ba624f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:18:41.437977 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a8aefd-1694-4c8a-8d46-022125ba624f None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:18:41.447348 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 247/491] 10.222.0.49 () {56 vars in 1103 bytes} [Wed Jun 15 10:18:41 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:18:41.448325 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a8aefd-1694-4c8a-8d46-022125ba624f None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:18:41.482052 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=100 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:18:41.490756 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 245/492] 10.222.0.49 () {58 vars in 1023 bytes} [Wed Jun 15 10:18:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 10:18:41.491913 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4af7961-3432-4da5-ae4f-2f7c595d688e None None] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:18:41.515888 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:18:41.564336 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a8aefd-1694-4c8a-8d46-022125ba624f None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:18:41.601428 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4af7961-3432-4da5-ae4f-2f7c595d688e None None] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:18:41.602812 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c843ebc2-40be-4c5a-9723-3e768c2bfef9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:18:41.603907 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c843ebc2-40be-4c5a-9723-3e768c2bfef9 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:41.604946 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c843ebc2-40be-4c5a-9723-3e768c2bfef9 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:41.604946 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c843ebc2-40be-4c5a-9723-3e768c2bfef9 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:41.616245 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c843ebc2-40be-4c5a-9723-3e768c2bfef9 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:18:41.618004 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c843ebc2-40be-4c5a-9723-3e768c2bfef9 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:18:41.637275 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application [None req-c843ebc2-40be-4c5a-9723-3e768c2bfef9 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 15 10:18:41.637275 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:18:41.637275 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:18:41.637275 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:18:41.637275 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:18:41.637275 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:18:41.637275 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:18:41.637275 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:18:41.637275 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:18:41.637275 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:18:41.637275 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:18:41.637275 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:18:41.637275 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 15 10:18:41.637275 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 15 10:18:41.637275 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 15 10:18:41.637275 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 15 10:18:41.638263 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:18:41.638263 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:18:41.638263 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 15 10:18:41.638263 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 10:18:41.638263 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 15 10:18:41.638263 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 10:18:41.638263 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:18:41.638263 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:18:41.638263 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:18:41.638263 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:18:41.638263 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:18:41.638263 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application with Lock( Jun 15 10:18:41.638263 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:18:41.638263 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:18:41.638263 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:18:41.638263 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:18:41.638263 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:18:41.638263 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:18:41.639198 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:18:41.639198 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:18:41.639198 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 15 10:18:41.639198 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 15 10:18:41.639198 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 15 10:18:41.639198 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 15 10:18:41.639198 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 15 10:18:41.639198 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 15 10:18:41.639198 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 15 10:18:41.639198 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Jun 15 10:18:41.643130 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 248/493] 10.222.0.49 () {56 vars in 1093 bytes} [Wed Jun 15 10:18:41 2022] GET /identity/v3/users/nova => generated 82 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:18:41.644200 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c843ebc2-40be-4c5a-9723-3e768c2bfef9 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:18:41.646206 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f3b252-ee6c-48ea-8368-bb428a9765bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:18:41.647412 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f3b252-ee6c-48ea-8368-bb428a9765bc None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:41.647910 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f3b252-ee6c-48ea-8368-bb428a9765bc None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:41.648441 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f3b252-ee6c-48ea-8368-bb428a9765bc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:41.660820 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f3b252-ee6c-48ea-8368-bb428a9765bc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:18:41.662841 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f3b252-ee6c-48ea-8368-bb428a9765bc None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:18:41.682515 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application [None req-c5f3b252-ee6c-48ea-8368-bb428a9765bc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 10:18:41.682515 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:18:41.682515 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:18:41.682515 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:18:41.682515 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:18:41.682515 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:18:41.682515 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:18:41.682515 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:18:41.682515 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:18:41.682515 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:18:41.682515 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:18:41.682515 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:18:41.682515 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 10:18:41.682515 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 10:18:41.682515 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 10:18:41.683501 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 10:18:41.683501 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:18:41.683501 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:18:41.683501 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:18:41.683501 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:18:41.683501 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:18:41.683501 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:18:41.683501 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:18:41.683501 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application with Lock( Jun 15 10:18:41.683501 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:18:41.683501 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:18:41.683501 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:18:41.683501 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:18:41.683501 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:18:41.683501 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:18:41.683501 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:18:41.683501 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:18:41.684429 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 10:18:41.684429 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 10:18:41.684429 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 10:18:41.684429 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application Jun 15 10:18:41.686666 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 246/494] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Jun 15 10:18:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:18:41.687728 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f3b252-ee6c-48ea-8368-bb428a9765bc None admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:18:41.737327 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=100 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:18:41.754206 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c843ebc2-40be-4c5a-9723-3e768c2bfef9 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:18:41.776545 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:18:41.796280 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f3b252-ee6c-48ea-8368-bb428a9765bc None admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:18:41.797927 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d4bef1-0d0c-47e7-9403-6cee8780c4f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:18:41.798796 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d4bef1-0d0c-47e7-9403-6cee8780c4f0 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:41.799113 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d4bef1-0d0c-47e7-9403-6cee8780c4f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:41.799397 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d4bef1-0d0c-47e7-9403-6cee8780c4f0 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:41.810509 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d4bef1-0d0c-47e7-9403-6cee8780c4f0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 10:18:41.818314 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application [None req-54d4bef1-0d0c-47e7-9403-6cee8780c4f0 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 15 10:18:41.818314 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:18:41.818314 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:18:41.818314 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:18:41.818314 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:18:41.818314 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:18:41.818314 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:18:41.818314 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:18:41.818314 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:18:41.818314 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:18:41.818314 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:18:41.818314 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:18:41.818314 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 10:18:41.818314 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 10:18:41.818314 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 10:18:41.818314 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 10:18:41.819225 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 10:18:41.819225 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 10:18:41.819225 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 10:18:41.819225 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 10:18:41.819225 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:18:41.819225 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:18:41.819225 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:18:41.819225 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:18:41.819225 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:18:41.819225 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:18:41.819225 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:18:41.819225 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application with Lock( Jun 15 10:18:41.819225 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:18:41.819225 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:18:41.819225 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:18:41.819225 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:18:41.820452 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:18:41.820452 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:18:41.820452 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:18:41.820452 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:18:41.820452 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 10:18:41.820452 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 10:18:41.820452 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 10:18:41.820452 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 10:18:41.820452 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 10:18:41.820452 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 10:18:41.820452 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 15 10:18:41.820452 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Jun 15 10:18:41.820452 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 249/495] 10.222.0.49 () {56 vars in 1102 bytes} [Wed Jun 15 10:18:41 2022] GET /identity/v3/roles/service => generated 85 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:18:41.821300 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0286dbbc-a1ce-4cbf-b904-cb60db1b4740 None None] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:41.821300 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0286dbbc-a1ce-4cbf-b904-cb60db1b4740 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:41.821427 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0286dbbc-a1ce-4cbf-b904-cb60db1b4740 None None] PATH_INFO: `/` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:41.822787 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 250/496] 10.222.0.49 () {54 vars in 926 bytes} [Wed Jun 15 10:18:41 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:18:41.823351 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0286dbbc-a1ce-4cbf-b904-cb60db1b4740 None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:18:41.843757 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2c293b-4b22-4583-9e8c-41697d984462 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:18:41.844642 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2c293b-4b22-4583-9e8c-41697d984462 None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:41.845100 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2c293b-4b22-4583-9e8c-41697d984462 None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:41.845523 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2c293b-4b22-4583-9e8c-41697d984462 None admin] PATH_INFO: `/v3/users` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:41.846626 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a2c293b-4b22-4583-9e8c-41697d984462 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:18:41.846946 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a2c293b-4b22-4583-9e8c-41697d984462 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:18:41.848417 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a2c293b-4b22-4583-9e8c-41697d984462 None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:18:41.892925 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 247/497] 10.222.0.49 () {56 vars in 1133 bytes} [Wed Jun 15 10:18:41 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:18:41.892925 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2c293b-4b22-4583-9e8c-41697d984462 None admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:18:41.930253 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0286dbbc-a1ce-4cbf-b904-cb60db1b4740 None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:18:41.966564 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28013e2b-ce21-4431-8d95-0781c9f30d59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:18:41.967267 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28013e2b-ce21-4431-8d95-0781c9f30d59 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:41.967556 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28013e2b-ce21-4431-8d95-0781c9f30d59 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:41.967856 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28013e2b-ce21-4431-8d95-0781c9f30d59 None admin] PATH_INFO: `/v3/domains` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:41.968560 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28013e2b-ce21-4431-8d95-0781c9f30d59 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:18:41.968872 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28013e2b-ce21-4431-8d95-0781c9f30d59 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:18:41.970270 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28013e2b-ce21-4431-8d95-0781c9f30d59 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:18:41.986694 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 251/498] 10.222.0.49 () {56 vars in 1109 bytes} [Wed Jun 15 10:18:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:18:41.986694 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6aa17f-6b05-42cb-aaa7-c62d841cdc2c None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:41.986694 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6aa17f-6b05-42cb-aaa7-c62d841cdc2c None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:41.986694 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6aa17f-6b05-42cb-aaa7-c62d841cdc2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:41.998156 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2c293b-4b22-4583-9e8c-41697d984462 None admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:18:42.002082 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-ec6aa17f-6b05-42cb-aaa7-c62d841cdc2c None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:18:42.003325 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-ec6aa17f-6b05-42cb-aaa7-c62d841cdc2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:18:42.005591 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=100 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:18:42.035045 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952e7631-ab8e-4184-9c9b-31f3340d6b46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:18:42.035277 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:18:42.035745 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952e7631-ab8e-4184-9c9b-31f3340d6b46 None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:42.036037 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952e7631-ab8e-4184-9c9b-31f3340d6b46 None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:42.036336 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952e7631-ab8e-4184-9c9b-31f3340d6b46 None admin] PATH_INFO: `/v3/roles` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:42.037137 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-952e7631-ab8e-4184-9c9b-31f3340d6b46 None admin] RBAC: Adding query filter params (name=service) {{(pid=80119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:18:42.037474 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-952e7631-ab8e-4184-9c9b-31f3340d6b46 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:18:42.038900 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-952e7631-ab8e-4184-9c9b-31f3340d6b46 None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:18:42.051728 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 248/499] 10.222.0.49 () {56 vars in 1103 bytes} [Wed Jun 15 10:18:41 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:18:42.051728 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952e7631-ab8e-4184-9c9b-31f3340d6b46 None admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:18:42.079309 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 252/500] 10.222.0.49 () {58 vars in 1023 bytes} [Wed Jun 15 10:18:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 10:18:42.080149 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6aa17f-6b05-42cb-aaa7-c62d841cdc2c None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:18:42.165325 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952e7631-ab8e-4184-9c9b-31f3340d6b46 None admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:18:42.199294 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bac840c-7426-4cca-9d97-23f2b89df1ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:18:42.200136 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bac840c-7426-4cca-9d97-23f2b89df1ee None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:42.200429 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bac840c-7426-4cca-9d97-23f2b89df1ee None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:42.200710 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bac840c-7426-4cca-9d97-23f2b89df1ee None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:42.202681 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6aa17f-6b05-42cb-aaa7-c62d841cdc2c None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:18:42.234305 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=100 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:18:42.236946 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0cca03-508e-411f-89c2-e47115d6a738 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:18:42.237740 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0cca03-508e-411f-89c2-e47115d6a738 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:42.238020 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0cca03-508e-411f-89c2-e47115d6a738 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:42.238292 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0cca03-508e-411f-89c2-e47115d6a738 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:42.245715 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0cca03-508e-411f-89c2-e47115d6a738 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:18:42.247749 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0cca03-508e-411f-89c2-e47115d6a738 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:18:42.262083 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application [None req-fd0cca03-508e-411f-89c2-e47115d6a738 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jun 15 10:18:42.262083 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:18:42.262083 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:18:42.262083 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:18:42.262083 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:18:42.262083 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:18:42.262083 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:18:42.262083 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:18:42.262083 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:18:42.262083 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:18:42.262083 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:18:42.262083 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:18:42.262083 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 15 10:18:42.262083 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 15 10:18:42.262083 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 15 10:18:42.262083 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 15 10:18:42.263191 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:18:42.263191 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:18:42.263191 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 15 10:18:42.263191 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 10:18:42.263191 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 15 10:18:42.263191 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 10:18:42.263191 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:18:42.263191 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:18:42.263191 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:18:42.263191 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:18:42.263191 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:18:42.263191 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application with Lock( Jun 15 10:18:42.263191 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:18:42.263191 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:18:42.263191 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:18:42.263191 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:18:42.263191 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:18:42.263191 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:18:42.268709 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:18:42.268709 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:18:42.268709 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 15 10:18:42.268709 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 15 10:18:42.268709 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 15 10:18:42.268709 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 15 10:18:42.268709 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 15 10:18:42.268709 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 15 10:18:42.268709 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jun 15 10:18:42.268709 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Jun 15 10:18:42.268709 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 253/501] 10.222.0.49 () {56 vars in 1099 bytes} [Wed Jun 15 10:18:42 2022] GET /identity/v3/users/glance => generated 84 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:18:42.268709 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0cca03-508e-411f-89c2-e47115d6a738 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:18:42.272826 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bac840c-7426-4cca-9d97-23f2b89df1ee None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:18:42.274306 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bac840c-7426-4cca-9d97-23f2b89df1ee None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:18:42.290972 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:18:42.360951 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0cca03-508e-411f-89c2-e47115d6a738 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:18:42.373023 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 249/502] 10.222.0.49 () {58 vars in 1282 bytes} [Wed Jun 15 10:18:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 15 10:18:42.373850 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bac840c-7426-4cca-9d97-23f2b89df1ee None admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:18:42.398248 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4a2433-dec5-4d3a-853c-2cc2e545082f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:18:42.399597 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4a2433-dec5-4d3a-853c-2cc2e545082f None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:42.399879 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4a2433-dec5-4d3a-853c-2cc2e545082f None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:42.400154 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4a2433-dec5-4d3a-853c-2cc2e545082f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:42.410088 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a4a2433-dec5-4d3a-853c-2cc2e545082f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:18:42.411568 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a4a2433-dec5-4d3a-853c-2cc2e545082f None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:18:42.429349 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application [None req-3a4a2433-dec5-4d3a-853c-2cc2e545082f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 10:18:42.429349 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:18:42.429349 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:18:42.429349 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:18:42.429349 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:18:42.429349 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:18:42.429349 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:18:42.429349 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:18:42.429349 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:18:42.429349 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:18:42.429349 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:18:42.429349 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:18:42.429349 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 10:18:42.429349 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 10:18:42.429349 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 10:18:42.430227 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 10:18:42.430227 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:18:42.430227 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:18:42.430227 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:18:42.430227 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:18:42.430227 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:18:42.430227 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:18:42.430227 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:18:42.430227 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application with Lock( Jun 15 10:18:42.430227 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:18:42.430227 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:18:42.430227 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:18:42.430227 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:18:42.430227 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:18:42.430227 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:18:42.430227 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:18:42.430227 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:18:42.431069 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 10:18:42.431069 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 10:18:42.431069 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 10:18:42.431069 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Jun 15 10:18:42.431069 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 254/503] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Jun 15 10:18:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:18:42.432017 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12689dd-fd38-446b-ad50-d35bbe85c088 None None] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:42.432303 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12689dd-fd38-446b-ad50-d35bbe85c088 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:42.432577 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12689dd-fd38-446b-ad50-d35bbe85c088 None None] PATH_INFO: `/` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:42.434439 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 255/504] 10.222.0.49 () {54 vars in 926 bytes} [Wed Jun 15 10:18:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:18:42.435218 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12689dd-fd38-446b-ad50-d35bbe85c088 None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:18:42.441504 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=100 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:18:42.475163 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bac840c-7426-4cca-9d97-23f2b89df1ee None admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:18:42.508538 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b372edae-adbc-4a04-92a4-5f17ccf4dea0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:18:42.509328 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b372edae-adbc-4a04-92a4-5f17ccf4dea0 None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:42.509622 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b372edae-adbc-4a04-92a4-5f17ccf4dea0 None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:42.509902 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b372edae-adbc-4a04-92a4-5f17ccf4dea0 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:42.516435 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:18:42.520432 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b372edae-adbc-4a04-92a4-5f17ccf4dea0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 10:18:42.528785 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application [None req-b372edae-adbc-4a04-92a4-5f17ccf4dea0 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 15 10:18:42.528785 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:18:42.528785 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:18:42.528785 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:18:42.528785 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:18:42.528785 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:18:42.528785 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:18:42.528785 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:18:42.528785 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:18:42.528785 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:18:42.528785 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:18:42.528785 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:18:42.528785 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 10:18:42.528785 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 10:18:42.528785 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 10:18:42.528785 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 10:18:42.529678 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 10:18:42.529678 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 10:18:42.529678 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 10:18:42.529678 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 10:18:42.529678 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:18:42.529678 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:18:42.529678 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:18:42.529678 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:18:42.529678 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:18:42.529678 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:18:42.529678 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:18:42.529678 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application with Lock( Jun 15 10:18:42.529678 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:18:42.529678 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:18:42.529678 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:18:42.529678 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:18:42.530748 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:18:42.530748 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:18:42.530748 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:18:42.530748 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:18:42.530748 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 10:18:42.530748 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 10:18:42.530748 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 10:18:42.530748 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 10:18:42.530748 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 10:18:42.530748 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 10:18:42.530748 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 15 10:18:42.530748 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application Jun 15 10:18:42.530748 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 250/505] 10.222.0.49 () {56 vars in 1102 bytes} [Wed Jun 15 10:18:42 2022] GET /identity/v3/roles/service => generated 85 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:18:42.532887 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b372edae-adbc-4a04-92a4-5f17ccf4dea0 None admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:18:42.536307 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12689dd-fd38-446b-ad50-d35bbe85c088 None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:18:42.569621 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607b26ef-daa0-4408-80c6-482078264bb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:18:42.570282 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607b26ef-daa0-4408-80c6-482078264bb9 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:42.570564 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607b26ef-daa0-4408-80c6-482078264bb9 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:42.570838 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607b26ef-daa0-4408-80c6-482078264bb9 None admin] PATH_INFO: `/v3/users` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:42.571696 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607b26ef-daa0-4408-80c6-482078264bb9 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:18:42.572007 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607b26ef-daa0-4408-80c6-482078264bb9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:18:42.573437 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607b26ef-daa0-4408-80c6-482078264bb9 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:18:42.609528 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 256/506] 10.222.0.49 () {56 vars in 1137 bytes} [Wed Jun 15 10:18:42 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:18:42.610329 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607b26ef-daa0-4408-80c6-482078264bb9 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:18:42.635033 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b372edae-adbc-4a04-92a4-5f17ccf4dea0 None admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:18:42.666621 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=100 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:18:42.675890 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce725f26-44e4-4680-a013-6244948e277f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:18:42.676944 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce725f26-44e4-4680-a013-6244948e277f None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:42.677229 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce725f26-44e4-4680-a013-6244948e277f None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:42.677506 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce725f26-44e4-4680-a013-6244948e277f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:42.687838 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce725f26-44e4-4680-a013-6244948e277f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:18:42.689298 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce725f26-44e4-4680-a013-6244948e277f None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:18:42.705850 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application [None req-ce725f26-44e4-4680-a013-6244948e277f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 10:18:42.705850 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:18:42.705850 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:18:42.705850 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:18:42.705850 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:18:42.705850 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:18:42.705850 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:18:42.705850 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:18:42.705850 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:18:42.705850 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:18:42.705850 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:18:42.705850 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:18:42.705850 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 10:18:42.705850 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 10:18:42.705850 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 10:18:42.706712 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 10:18:42.706712 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:18:42.706712 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:18:42.706712 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:18:42.706712 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:18:42.706712 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:18:42.706712 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:18:42.706712 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:18:42.706712 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application with Lock( Jun 15 10:18:42.706712 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:18:42.706712 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:18:42.706712 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:18:42.706712 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:18:42.706712 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:18:42.706712 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:18:42.706712 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:18:42.706712 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:18:42.707835 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 10:18:42.707835 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 10:18:42.707835 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 10:18:42.707835 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application Jun 15 10:18:42.708238 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607b26ef-daa0-4408-80c6-482078264bb9 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:18:42.708337 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 251/507] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Jun 15 10:18:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:18:42.708337 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28226df-889e-437b-b806-67ae9a1538d7 None None] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:42.708484 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28226df-889e-437b-b806-67ae9a1538d7 None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:42.708734 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28226df-889e-437b-b806-67ae9a1538d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:42.725004 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.auth.core [None req-c28226df-889e-437b-b806-67ae9a1538d7 None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:18:42.726275 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-c28226df-889e-437b-b806-67ae9a1538d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:18:42.741171 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930c236a-534e-4c9e-8135-c5fadc06f3da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:18:42.741835 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930c236a-534e-4c9e-8135-c5fadc06f3da None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:42.742120 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930c236a-534e-4c9e-8135-c5fadc06f3da None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:42.742396 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930c236a-534e-4c9e-8135-c5fadc06f3da None admin] PATH_INFO: `/v3/domains` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:42.743140 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930c236a-534e-4c9e-8135-c5fadc06f3da None admin] RBAC: Adding query filter params (name=Default) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:18:42.743448 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930c236a-534e-4c9e-8135-c5fadc06f3da None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:18:42.744815 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930c236a-534e-4c9e-8135-c5fadc06f3da None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:18:42.751193 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:18:42.756614 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 257/508] 10.222.0.49 () {56 vars in 1109 bytes} [Wed Jun 15 10:18:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:18:42.757400 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930c236a-534e-4c9e-8135-c5fadc06f3da None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:18:42.794163 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 252/509] 10.222.0.49 () {58 vars in 1023 bytes} [Wed Jun 15 10:18:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 10:18:42.794804 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28226df-889e-437b-b806-67ae9a1538d7 None None] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:18:42.867731 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930c236a-534e-4c9e-8135-c5fadc06f3da None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:18:42.894225 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28226df-889e-437b-b806-67ae9a1538d7 None None] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:18:42.902165 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90eda729-16fe-4779-b3f1-0023ebc14b70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:18:42.903360 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90eda729-16fe-4779-b3f1-0023ebc14b70 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:42.903650 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90eda729-16fe-4779-b3f1-0023ebc14b70 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:42.903929 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90eda729-16fe-4779-b3f1-0023ebc14b70 None admin] PATH_INFO: `/v3/roles` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:42.904679 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90eda729-16fe-4779-b3f1-0023ebc14b70 None admin] RBAC: Adding query filter params (name=service) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:18:42.904993 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90eda729-16fe-4779-b3f1-0023ebc14b70 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:18:42.905778 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=100 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:18:42.906369 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90eda729-16fe-4779-b3f1-0023ebc14b70 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:18:42.914442 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 258/510] 10.222.0.49 () {56 vars in 1103 bytes} [Wed Jun 15 10:18:42 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:18:42.915258 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90eda729-16fe-4779-b3f1-0023ebc14b70 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:18:42.931338 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05328865-16a2-4d3d-92ec-b5d687d925af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:18:42.931995 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05328865-16a2-4d3d-92ec-b5d687d925af None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:42.932282 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05328865-16a2-4d3d-92ec-b5d687d925af None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:42.932559 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05328865-16a2-4d3d-92ec-b5d687d925af None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:42.973607 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:18:42.999905 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05328865-16a2-4d3d-92ec-b5d687d925af None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:18:43.001311 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05328865-16a2-4d3d-92ec-b5d687d925af None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:18:43.014101 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90eda729-16fe-4779-b3f1-0023ebc14b70 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:18:43.047097 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c288314d-842c-48f8-ad9e-dd4dc5b130a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:18:43.047781 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c288314d-842c-48f8-ad9e-dd4dc5b130a6 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:43.048068 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c288314d-842c-48f8-ad9e-dd4dc5b130a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:43.048357 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c288314d-842c-48f8-ad9e-dd4dc5b130a6 None admin] PATH_INFO: `/v3/domains` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:43.049070 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c288314d-842c-48f8-ad9e-dd4dc5b130a6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:18:43.049379 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c288314d-842c-48f8-ad9e-dd4dc5b130a6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:18:43.050747 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c288314d-842c-48f8-ad9e-dd4dc5b130a6 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:18:43.062484 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 259/511] 10.222.0.49 () {56 vars in 1109 bytes} [Wed Jun 15 10:18:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:18:43.063286 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c288314d-842c-48f8-ad9e-dd4dc5b130a6 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:18:43.101620 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 253/512] 10.222.0.49 () {58 vars in 1282 bytes} [Wed Jun 15 10:18:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 15 10:18:43.102444 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05328865-16a2-4d3d-92ec-b5d687d925af None admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:18:43.113898 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=100 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:18:43.160576 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c288314d-842c-48f8-ad9e-dd4dc5b130a6 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:18:43.181334 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:18:43.196126 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b8af24-53a0-4ffe-8640-b43149e87264 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:18:43.197104 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b8af24-53a0-4ffe-8640-b43149e87264 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:43.197393 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b8af24-53a0-4ffe-8640-b43149e87264 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:43.197673 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b8af24-53a0-4ffe-8640-b43149e87264 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:43.201338 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05328865-16a2-4d3d-92ec-b5d687d925af None admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:18:43.205129 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b8af24-53a0-4ffe-8640-b43149e87264 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:18:43.206579 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b8af24-53a0-4ffe-8640-b43149e87264 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:18:43.221353 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application [None req-f3b8af24-53a0-4ffe-8640-b43149e87264 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 15 10:18:43.221353 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:18:43.221353 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:18:43.221353 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:18:43.221353 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:18:43.221353 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:18:43.221353 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:18:43.221353 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:18:43.221353 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:18:43.221353 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:18:43.221353 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:18:43.221353 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:18:43.221353 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 15 10:18:43.221353 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 15 10:18:43.221353 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 15 10:18:43.221353 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 15 10:18:43.222215 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:18:43.222215 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:18:43.222215 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 15 10:18:43.222215 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 10:18:43.222215 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 15 10:18:43.222215 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 10:18:43.222215 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:18:43.222215 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:18:43.222215 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:18:43.222215 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:18:43.222215 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:18:43.222215 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application with Lock( Jun 15 10:18:43.222215 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:18:43.222215 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:18:43.222215 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:18:43.222215 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:18:43.222215 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:18:43.222215 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:18:43.223361 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:18:43.223361 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:18:43.223361 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 15 10:18:43.223361 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 15 10:18:43.223361 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 15 10:18:43.223361 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 15 10:18:43.223361 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 15 10:18:43.223361 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 15 10:18:43.223361 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 15 10:18:43.223361 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Jun 15 10:18:43.223361 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 260/513] 10.222.0.49 () {56 vars in 1099 bytes} [Wed Jun 15 10:18:43 2022] GET /identity/v3/users/cinder => generated 84 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:18:43.223361 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b8af24-53a0-4ffe-8640-b43149e87264 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:18:43.238605 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e08995a-ba7c-4061-a78e-5cf246c87810 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:18:43.239583 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e08995a-ba7c-4061-a78e-5cf246c87810 None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:43.239874 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e08995a-ba7c-4061-a78e-5cf246c87810 None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:43.240152 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e08995a-ba7c-4061-a78e-5cf246c87810 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:43.250535 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e08995a-ba7c-4061-a78e-5cf246c87810 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:18:43.251952 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e08995a-ba7c-4061-a78e-5cf246c87810 None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:18:43.269061 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application [None req-0e08995a-ba7c-4061-a78e-5cf246c87810 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 10:18:43.269061 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:18:43.269061 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:18:43.269061 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:18:43.269061 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:18:43.269061 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:18:43.269061 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:18:43.269061 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:18:43.269061 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:18:43.269061 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:18:43.269061 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:18:43.269061 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:18:43.269061 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 10:18:43.269061 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 10:18:43.269061 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 10:18:43.269904 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 10:18:43.269904 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:18:43.269904 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:18:43.269904 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:18:43.269904 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:18:43.269904 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:18:43.269904 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:18:43.269904 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:18:43.269904 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application with Lock( Jun 15 10:18:43.269904 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:18:43.269904 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:18:43.269904 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:18:43.269904 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:18:43.269904 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:18:43.269904 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:18:43.269904 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:18:43.269904 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:18:43.270773 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 10:18:43.270773 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 10:18:43.270773 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 10:18:43.270773 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application Jun 15 10:18:43.270773 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 254/514] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Jun 15 10:18:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:18:43.271335 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e08995a-ba7c-4061-a78e-5cf246c87810 None admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:18:43.322367 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b8af24-53a0-4ffe-8640-b43149e87264 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:18:43.334188 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=100 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:18:43.355789 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5fd9dfb-fec5-4195-b635-6ba755b3ecc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:18:43.356584 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fd9dfb-fec5-4195-b635-6ba755b3ecc7 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:43.356869 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fd9dfb-fec5-4195-b635-6ba755b3ecc7 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:43.357142 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fd9dfb-fec5-4195-b635-6ba755b3ecc7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:43.366969 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5fd9dfb-fec5-4195-b635-6ba755b3ecc7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:18:43.368485 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5fd9dfb-fec5-4195-b635-6ba755b3ecc7 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:18:43.373624 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e08995a-ba7c-4061-a78e-5cf246c87810 None admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:18:43.385798 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application [None req-a5fd9dfb-fec5-4195-b635-6ba755b3ecc7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 10:18:43.385798 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:18:43.385798 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:18:43.385798 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:18:43.385798 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:18:43.385798 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:18:43.385798 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:18:43.385798 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:18:43.385798 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:18:43.385798 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:18:43.385798 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:18:43.385798 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:18:43.385798 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 10:18:43.385798 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 10:18:43.385798 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 10:18:43.387281 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 10:18:43.387281 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:18:43.387281 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:18:43.387281 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:18:43.387281 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:18:43.387281 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:18:43.387281 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:18:43.387281 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:18:43.387281 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application with Lock( Jun 15 10:18:43.387281 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:18:43.387281 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:18:43.387281 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:18:43.387281 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:18:43.387281 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:18:43.387281 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:18:43.387281 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:18:43.387281 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:18:43.392831 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 10:18:43.392831 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 10:18:43.392831 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 10:18:43.392831 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Jun 15 10:18:43.392831 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 261/515] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Jun 15 10:18:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:18:43.392831 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5fd9dfb-fec5-4195-b635-6ba755b3ecc7 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:18:43.408076 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682ac33f-804b-4230-8384-becd62a874f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:18:43.408872 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682ac33f-804b-4230-8384-becd62a874f2 None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:43.409155 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682ac33f-804b-4230-8384-becd62a874f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:43.409433 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682ac33f-804b-4230-8384-becd62a874f2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:43.416099 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-682ac33f-804b-4230-8384-becd62a874f2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:18:43.417567 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-682ac33f-804b-4230-8384-becd62a874f2 None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:18:43.420358 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:18:43.434039 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application [None req-682ac33f-804b-4230-8384-becd62a874f2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 10:18:43.434039 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:18:43.434039 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:18:43.434039 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:18:43.434039 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:18:43.434039 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:18:43.434039 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:18:43.434039 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:18:43.434039 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:18:43.434039 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:18:43.434039 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:18:43.434039 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:18:43.434039 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 15 10:18:43.434039 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 15 10:18:43.434039 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 15 10:18:43.434930 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 15 10:18:43.434930 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:18:43.434930 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:18:43.434930 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:18:43.434930 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:18:43.434930 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:18:43.434930 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:18:43.434930 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:18:43.434930 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application with Lock( Jun 15 10:18:43.434930 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:18:43.434930 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:18:43.434930 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:18:43.434930 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:18:43.434930 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:18:43.434930 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:18:43.434930 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:18:43.434930 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:18:43.435826 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 15 10:18:43.435826 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 15 10:18:43.435826 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 15 10:18:43.435826 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 15 10:18:43.435826 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 15 10:18:43.435826 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 15 10:18:43.435826 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 10:18:43.435826 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application Jun 15 10:18:43.435826 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 255/516] 10.222.0.49 () {56 vars in 1111 bytes} [Wed Jun 15 10:18:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:18:43.436509 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682ac33f-804b-4230-8384-becd62a874f2 None admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:18:43.491525 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5fd9dfb-fec5-4195-b635-6ba755b3ecc7 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:18:43.525646 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96afa181-24df-4049-b388-d86f7b7dc8fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:18:43.526374 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96afa181-24df-4049-b388-d86f7b7dc8fc None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:43.526662 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96afa181-24df-4049-b388-d86f7b7dc8fc None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:43.526940 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96afa181-24df-4049-b388-d86f7b7dc8fc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:43.534387 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682ac33f-804b-4230-8384-becd62a874f2 None admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:18:43.536787 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96afa181-24df-4049-b388-d86f7b7dc8fc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:18:43.538239 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96afa181-24df-4049-b388-d86f7b7dc8fc None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:18:43.555466 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application [None req-96afa181-24df-4049-b388-d86f7b7dc8fc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 10:18:43.555466 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:18:43.555466 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:18:43.555466 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:18:43.555466 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:18:43.555466 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:18:43.555466 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:18:43.555466 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:18:43.555466 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:18:43.555466 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:18:43.555466 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:18:43.555466 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:18:43.555466 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 10:18:43.555466 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 10:18:43.555466 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 10:18:43.556311 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 10:18:43.556311 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:18:43.556311 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:18:43.556311 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:18:43.556311 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:18:43.556311 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:18:43.556311 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:18:43.556311 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:18:43.556311 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application with Lock( Jun 15 10:18:43.556311 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:18:43.556311 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:18:43.556311 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:18:43.556311 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:18:43.556311 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:18:43.556311 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:18:43.556311 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:18:43.556311 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:18:43.562676 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 10:18:43.562676 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 10:18:43.562676 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 10:18:43.562676 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Jun 15 10:18:43.562676 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 262/517] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Jun 15 10:18:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:18:43.562951 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=100 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:18:43.563021 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96afa181-24df-4049-b388-d86f7b7dc8fc None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:18:43.570038 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6264cd9d-47b9-4258-83f6-97bdf9000de6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:18:43.570703 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6264cd9d-47b9-4258-83f6-97bdf9000de6 None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:43.570981 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6264cd9d-47b9-4258-83f6-97bdf9000de6 None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:43.571286 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6264cd9d-47b9-4258-83f6-97bdf9000de6 None admin] PATH_INFO: `/v3/users` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:43.572117 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6264cd9d-47b9-4258-83f6-97bdf9000de6 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:18:43.572420 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6264cd9d-47b9-4258-83f6-97bdf9000de6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:18:43.573876 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6264cd9d-47b9-4258-83f6-97bdf9000de6 None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:18:43.591283 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 256/518] 10.222.0.49 () {56 vars in 1137 bytes} [Wed Jun 15 10:18:43 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:18:43.592061 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6264cd9d-47b9-4258-83f6-97bdf9000de6 None admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:18:43.644020 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:18:43.659280 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96afa181-24df-4049-b388-d86f7b7dc8fc None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:18:43.693029 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db332308-6b1a-474a-bb18-7ccd4909ae60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:18:43.693218 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6264cd9d-47b9-4258-83f6-97bdf9000de6 None admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:18:43.693681 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db332308-6b1a-474a-bb18-7ccd4909ae60 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:43.693963 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db332308-6b1a-474a-bb18-7ccd4909ae60 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:43.694241 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db332308-6b1a-474a-bb18-7ccd4909ae60 None admin] PATH_INFO: `/v3/domains` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:43.694955 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db332308-6b1a-474a-bb18-7ccd4909ae60 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:18:43.695287 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db332308-6b1a-474a-bb18-7ccd4909ae60 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:18:43.696661 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db332308-6b1a-474a-bb18-7ccd4909ae60 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:18:43.707477 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 263/519] 10.222.0.49 () {56 vars in 1109 bytes} [Wed Jun 15 10:18:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:18:43.708249 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db332308-6b1a-474a-bb18-7ccd4909ae60 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:18:43.731274 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd7cb73-1310-4b19-849d-66ad586019fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:18:43.731935 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd7cb73-1310-4b19-849d-66ad586019fc None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:43.732218 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd7cb73-1310-4b19-849d-66ad586019fc None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:43.732491 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd7cb73-1310-4b19-849d-66ad586019fc None admin] PATH_INFO: `/v3/domains` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:43.733201 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd7cb73-1310-4b19-849d-66ad586019fc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:18:43.733506 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd7cb73-1310-4b19-849d-66ad586019fc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:18:43.734882 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd7cb73-1310-4b19-849d-66ad586019fc None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:18:43.746951 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 257/520] 10.222.0.49 () {56 vars in 1109 bytes} [Wed Jun 15 10:18:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:18:43.747860 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd7cb73-1310-4b19-849d-66ad586019fc None admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:18:43.794266 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=100 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:18:43.807737 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db332308-6b1a-474a-bb18-7ccd4909ae60 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:18:43.840321 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47403e9e-13d5-4fae-9768-f3c3460fb837 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:18:43.840978 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47403e9e-13d5-4fae-9768-f3c3460fb837 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:43.841267 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47403e9e-13d5-4fae-9768-f3c3460fb837 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:43.841545 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47403e9e-13d5-4fae-9768-f3c3460fb837 None admin] PATH_INFO: `/v3/projects` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:43.842324 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47403e9e-13d5-4fae-9768-f3c3460fb837 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:18:43.842632 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47403e9e-13d5-4fae-9768-f3c3460fb837 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:18:43.844092 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47403e9e-13d5-4fae-9768-f3c3460fb837 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:18:43.849594 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd7cb73-1310-4b19-849d-66ad586019fc None admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:18:43.862246 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 264/521] 10.222.0.49 () {56 vars in 1148 bytes} [Wed Jun 15 10:18:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:18:43.863042 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47403e9e-13d5-4fae-9768-f3c3460fb837 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:18:43.866446 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:18:43.884262 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de8de37-be61-49bd-8d59-e99e6d7879c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:18:43.884925 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de8de37-be61-49bd-8d59-e99e6d7879c3 None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:43.885211 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de8de37-be61-49bd-8d59-e99e6d7879c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:43.885498 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de8de37-be61-49bd-8d59-e99e6d7879c3 None admin] PATH_INFO: `/v3/domains` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:43.886211 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de8de37-be61-49bd-8d59-e99e6d7879c3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:18:43.886520 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de8de37-be61-49bd-8d59-e99e6d7879c3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:18:43.887917 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de8de37-be61-49bd-8d59-e99e6d7879c3 None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:18:43.899188 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 258/522] 10.222.0.49 () {56 vars in 1109 bytes} [Wed Jun 15 10:18:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:18:43.899188 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de8de37-be61-49bd-8d59-e99e6d7879c3 None admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:18:43.961322 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47403e9e-13d5-4fae-9768-f3c3460fb837 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:18:43.994481 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90cb1d26-3cee-4e38-ba19-c65f02fff186 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:18:43.995151 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90cb1d26-3cee-4e38-ba19-c65f02fff186 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:43.995299 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=100 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:18:43.995436 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90cb1d26-3cee-4e38-ba19-c65f02fff186 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:43.995754 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90cb1d26-3cee-4e38-ba19-c65f02fff186 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:43.998832 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de8de37-be61-49bd-8d59-e99e6d7879c3 None admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:18:44.032490 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe28b0a5-5053-4088-b3ab-c7a2af7d7ecf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:18:44.033144 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe28b0a5-5053-4088-b3ab-c7a2af7d7ecf None admin] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:44.033429 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe28b0a5-5053-4088-b3ab-c7a2af7d7ecf None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:44.033704 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe28b0a5-5053-4088-b3ab-c7a2af7d7ecf None admin] PATH_INFO: `/v3/users` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:44.034392 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe28b0a5-5053-4088-b3ab-c7a2af7d7ecf None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:18:44.035858 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe28b0a5-5053-4088-b3ab-c7a2af7d7ecf None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:18:44.055555 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 259/523] 10.222.0.49 () {60 vars in 1132 bytes} [Wed Jun 15 10:18:43 2022] POST /identity/v3/users => generated 247 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 10:18:44.056275 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe28b0a5-5053-4088-b3ab-c7a2af7d7ecf None admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:18:44.063693 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90cb1d26-3cee-4e38-ba19-c65f02fff186 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:18:44.065179 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90cb1d26-3cee-4e38-ba19-c65f02fff186 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:18:44.068384 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:18:44.178260 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 265/524] 10.222.0.49 () {58 vars in 1282 bytes} [Wed Jun 15 10:18:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 15 10:18:44.178671 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe28b0a5-5053-4088-b3ab-c7a2af7d7ecf None admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:18:44.179629 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90cb1d26-3cee-4e38-ba19-c65f02fff186 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:18:44.224310 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86483b1-addc-4b81-8f48-2c5d60f922a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:18:44.225118 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86483b1-addc-4b81-8f48-2c5d60f922a8 None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:44.225402 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86483b1-addc-4b81-8f48-2c5d60f922a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:44.225676 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86483b1-addc-4b81-8f48-2c5d60f922a8 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:44.234804 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c86483b1-addc-4b81-8f48-2c5d60f922a8 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:18:44.236315 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c86483b1-addc-4b81-8f48-2c5d60f922a8 None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:18:44.253938 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application [None req-c86483b1-addc-4b81-8f48-2c5d60f922a8 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jun 15 10:18:44.253938 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:18:44.253938 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:18:44.253938 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:18:44.253938 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:18:44.253938 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:18:44.253938 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:18:44.253938 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:18:44.253938 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:18:44.253938 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:18:44.253938 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:18:44.253938 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:18:44.253938 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 15 10:18:44.253938 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 15 10:18:44.253938 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 15 10:18:44.253938 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 15 10:18:44.254874 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:18:44.254874 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:18:44.254874 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 15 10:18:44.254874 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 10:18:44.254874 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 15 10:18:44.254874 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 10:18:44.254874 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:18:44.254874 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:18:44.254874 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:18:44.254874 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:18:44.254874 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:18:44.254874 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application with Lock( Jun 15 10:18:44.254874 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:18:44.254874 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:18:44.254874 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:18:44.254874 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:18:44.254874 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:18:44.254874 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:18:44.255830 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:18:44.255830 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:18:44.255830 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 15 10:18:44.255830 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 15 10:18:44.255830 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 15 10:18:44.255830 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 15 10:18:44.255830 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 15 10:18:44.255830 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 15 10:18:44.255830 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jun 15 10:18:44.255830 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application Jun 15 10:18:44.256400 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 260/525] 10.222.0.49 () {56 vars in 1102 bytes} [Wed Jun 15 10:18:44 2022] GET /identity/v3/users/neutron => generated 85 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:18:44.258995 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86483b1-addc-4b81-8f48-2c5d60f922a8 None admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:18:44.279829 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:18:44.295944 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90cb1d26-3cee-4e38-ba19-c65f02fff186 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:18:44.298976 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:18:44.333655 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0906b2e-b991-450f-b9ae-7d3112e66952 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:18:44.334243 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0906b2e-b991-450f-b9ae-7d3112e66952 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:44.334528 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0906b2e-b991-450f-b9ae-7d3112e66952 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:44.334800 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0906b2e-b991-450f-b9ae-7d3112e66952 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:44.335664 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0906b2e-b991-450f-b9ae-7d3112e66952 None admin] RBAC: Adding query filter params (role.id=16e76452bb5d4ce68a119ad3c9bde41c, scope.project.id=559b3e713f5844b6ae37151d56dd419f, user.id=5ac54528370d4747b6e8777fba9d0c82) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:18:44.335979 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0906b2e-b991-450f-b9ae-7d3112e66952 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:18:44.337406 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0906b2e-b991-450f-b9ae-7d3112e66952 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:18:44.351141 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 266/526] 10.222.0.49 () {56 vars in 1374 bytes} [Wed Jun 15 10:18:44 2022] GET /identity/v3/role_assignments?user.id=5ac54528370d4747b6e8777fba9d0c82&scope.project.id=559b3e713f5844b6ae37151d56dd419f&role.id=16e76452bb5d4ce68a119ad3c9bde41c => generated 259 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:18:44.351141 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0906b2e-b991-450f-b9ae-7d3112e66952 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:18:44.388836 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86483b1-addc-4b81-8f48-2c5d60f922a8 None admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:18:44.440335 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e685e0d6-589b-43fc-8b86-36ea4e097148 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:18:44.441984 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e685e0d6-589b-43fc-8b86-36ea4e097148 None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:44.442284 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e685e0d6-589b-43fc-8b86-36ea4e097148 None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:44.442563 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e685e0d6-589b-43fc-8b86-36ea4e097148 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:44.453620 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e685e0d6-589b-43fc-8b86-36ea4e097148 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:18:44.455437 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e685e0d6-589b-43fc-8b86-36ea4e097148 None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:18:44.462623 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0906b2e-b991-450f-b9ae-7d3112e66952 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:18:44.476703 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application [None req-e685e0d6-589b-43fc-8b86-36ea4e097148 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 10:18:44.476703 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:18:44.476703 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:18:44.476703 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:18:44.476703 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:18:44.476703 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:18:44.476703 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:18:44.476703 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:18:44.476703 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:18:44.476703 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:18:44.476703 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:18:44.476703 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:18:44.476703 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 15 10:18:44.476703 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 15 10:18:44.476703 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 15 10:18:44.477825 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 15 10:18:44.477825 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:18:44.477825 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:18:44.477825 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:18:44.477825 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:18:44.477825 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:18:44.477825 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:18:44.477825 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:18:44.477825 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application with Lock( Jun 15 10:18:44.477825 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:18:44.477825 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:18:44.477825 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:18:44.477825 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:18:44.477825 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:18:44.477825 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:18:44.477825 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:18:44.477825 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:18:44.478900 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 15 10:18:44.478900 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 15 10:18:44.478900 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 15 10:18:44.478900 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 15 10:18:44.478900 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 15 10:18:44.478900 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 15 10:18:44.478900 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 10:18:44.478900 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application Jun 15 10:18:44.478900 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 261/527] 10.222.0.49 () {56 vars in 1111 bytes} [Wed Jun 15 10:18:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:18:44.485049 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e685e0d6-589b-43fc-8b86-36ea4e097148 None admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:18:44.510217 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29aa813b-bfc7-4398-bfc4-24a0f1daefd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:18:44.512116 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29aa813b-bfc7-4398-bfc4-24a0f1daefd1 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:44.512681 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29aa813b-bfc7-4398-bfc4-24a0f1daefd1 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:44.514663 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29aa813b-bfc7-4398-bfc4-24a0f1daefd1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:44.528210 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29aa813b-bfc7-4398-bfc4-24a0f1daefd1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:18:44.529967 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29aa813b-bfc7-4398-bfc4-24a0f1daefd1 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:18:44.553672 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application [None req-29aa813b-bfc7-4398-bfc4-24a0f1daefd1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 10:18:44.553672 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:18:44.553672 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:18:44.553672 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:18:44.553672 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:18:44.553672 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:18:44.553672 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:18:44.553672 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:18:44.553672 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:18:44.553672 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:18:44.553672 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:18:44.553672 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:18:44.553672 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 10:18:44.553672 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 10:18:44.553672 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 10:18:44.554590 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 10:18:44.554590 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:18:44.554590 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:18:44.554590 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:18:44.554590 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:18:44.554590 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:18:44.554590 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:18:44.554590 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:18:44.554590 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application with Lock( Jun 15 10:18:44.554590 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:18:44.554590 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:18:44.554590 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:18:44.554590 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:18:44.554590 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:18:44.554590 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:18:44.554590 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:18:44.554590 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:18:44.555492 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 10:18:44.555492 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 10:18:44.555492 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 10:18:44.555492 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Jun 15 10:18:44.555492 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 267/528] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Jun 15 10:18:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:18:44.556386 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29aa813b-bfc7-4398-bfc4-24a0f1daefd1 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:18:44.584425 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:18:44.637548 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e685e0d6-589b-43fc-8b86-36ea4e097148 None admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:18:44.651532 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=100 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:18:44.673531 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29aa813b-bfc7-4398-bfc4-24a0f1daefd1 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:18:44.699724 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3644903e-6142-4556-91f5-e10153a05d85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:18:44.700463 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3644903e-6142-4556-91f5-e10153a05d85 None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:44.700752 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3644903e-6142-4556-91f5-e10153a05d85 None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:44.701027 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3644903e-6142-4556-91f5-e10153a05d85 None admin] PATH_INFO: `/v3/users` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:44.701864 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3644903e-6142-4556-91f5-e10153a05d85 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:18:44.702171 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3644903e-6142-4556-91f5-e10153a05d85 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:18:44.703675 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3644903e-6142-4556-91f5-e10153a05d85 None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:18:44.722960 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 262/529] 10.222.0.49 () {56 vars in 1139 bytes} [Wed Jun 15 10:18:44 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:18:44.727626 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3644903e-6142-4556-91f5-e10153a05d85 None admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:18:44.728470 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1cf3a9e-4f39-463d-b2f8-41d840c98608 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:18:44.729140 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cf3a9e-4f39-463d-b2f8-41d840c98608 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:44.729423 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cf3a9e-4f39-463d-b2f8-41d840c98608 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:44.729698 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cf3a9e-4f39-463d-b2f8-41d840c98608 None admin] PATH_INFO: `/v3/projects` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:44.730544 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1cf3a9e-4f39-463d-b2f8-41d840c98608 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:18:44.730850 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1cf3a9e-4f39-463d-b2f8-41d840c98608 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:18:44.732346 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1cf3a9e-4f39-463d-b2f8-41d840c98608 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:18:44.748532 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 268/530] 10.222.0.49 () {56 vars in 1148 bytes} [Wed Jun 15 10:18:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:18:44.749387 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1cf3a9e-4f39-463d-b2f8-41d840c98608 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:18:44.845039 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3644903e-6142-4556-91f5-e10153a05d85 None admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:18:44.865724 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:18:44.884321 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c66bf3-b1b0-495e-a489-9557acc9b08e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:18:44.885339 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c66bf3-b1b0-495e-a489-9557acc9b08e None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:44.885878 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c66bf3-b1b0-495e-a489-9557acc9b08e None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:44.886522 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c66bf3-b1b0-495e-a489-9557acc9b08e None admin] PATH_INFO: `/v3/domains` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:44.887983 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c66bf3-b1b0-495e-a489-9557acc9b08e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:18:44.888494 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c66bf3-b1b0-495e-a489-9557acc9b08e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:18:44.889791 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1cf3a9e-4f39-463d-b2f8-41d840c98608 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:18:44.890387 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c66bf3-b1b0-495e-a489-9557acc9b08e None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:18:44.893842 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=100 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:18:44.907610 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 263/531] 10.222.0.49 () {56 vars in 1109 bytes} [Wed Jun 15 10:18:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:18:44.908899 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c66bf3-b1b0-495e-a489-9557acc9b08e None admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:18:44.933817 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e06e408-8f86-4a0f-b549-746d1712e70f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:18:44.934933 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e06e408-8f86-4a0f-b549-746d1712e70f None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:44.935255 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e06e408-8f86-4a0f-b549-746d1712e70f None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:44.935539 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e06e408-8f86-4a0f-b549-746d1712e70f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:45.022265 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c66bf3-b1b0-495e-a489-9557acc9b08e None admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:18:45.024651 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e06e408-8f86-4a0f-b549-746d1712e70f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:18:45.026892 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e06e408-8f86-4a0f-b549-746d1712e70f None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:18:45.060226 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc34a6e-9f5c-4448-88d8-e2f2ccac3d72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:18:45.061475 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc34a6e-9f5c-4448-88d8-e2f2ccac3d72 None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:45.061965 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc34a6e-9f5c-4448-88d8-e2f2ccac3d72 None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:45.062508 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc34a6e-9f5c-4448-88d8-e2f2ccac3d72 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:45.063677 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc34a6e-9f5c-4448-88d8-e2f2ccac3d72 None admin] RBAC: Adding query filter params (role.id=16e76452bb5d4ce68a119ad3c9bde41c, scope.project.id=559b3e713f5844b6ae37151d56dd419f, user.id=91f2fe3be62e49548eae208e57bf2a93) {{(pid=80119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:18:45.064194 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc34a6e-9f5c-4448-88d8-e2f2ccac3d72 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:18:45.066065 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc34a6e-9f5c-4448-88d8-e2f2ccac3d72 None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:18:45.072261 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 264/532] 10.222.0.49 () {56 vars in 1374 bytes} [Wed Jun 15 10:18:44 2022] GET /identity/v3/role_assignments?user.id=91f2fe3be62e49548eae208e57bf2a93&scope.project.id=559b3e713f5844b6ae37151d56dd419f&role.id=16e76452bb5d4ce68a119ad3c9bde41c => generated 259 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:18:45.073348 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc34a6e-9f5c-4448-88d8-e2f2ccac3d72 None admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:18:45.167623 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:18:45.213676 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 269/533] 10.222.0.49 () {58 vars in 1282 bytes} [Wed Jun 15 10:18:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 464 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 15 10:18:45.219951 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=100 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:18:45.221768 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e06e408-8f86-4a0f-b549-746d1712e70f None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:18:45.291255 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc34a6e-9f5c-4448-88d8-e2f2ccac3d72 None admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:18:45.339025 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e06e408-8f86-4a0f-b549-746d1712e70f None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:18:45.343930 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ef80be-87e8-42c2-8e5a-881985edc729 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:18:45.344745 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ef80be-87e8-42c2-8e5a-881985edc729 None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:45.345028 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ef80be-87e8-42c2-8e5a-881985edc729 None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:45.345302 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ef80be-87e8-42c2-8e5a-881985edc729 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:45.358107 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ef80be-87e8-42c2-8e5a-881985edc729 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:18:45.360290 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ef80be-87e8-42c2-8e5a-881985edc729 None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:18:45.381801 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc635c8-df8e-4c46-8df7-9239084be354 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:18:45.382020 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc635c8-df8e-4c46-8df7-9239084be354 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:45.382289 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc635c8-df8e-4c46-8df7-9239084be354 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:45.382564 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc635c8-df8e-4c46-8df7-9239084be354 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:45.387626 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application [None req-65ef80be-87e8-42c2-8e5a-881985edc729 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 10:18:45.387626 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:18:45.387626 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:18:45.387626 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:18:45.387626 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:18:45.387626 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:18:45.387626 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:18:45.387626 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:18:45.387626 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:18:45.387626 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:18:45.387626 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:18:45.387626 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:18:45.387626 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 15 10:18:45.387626 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 15 10:18:45.387626 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 15 10:18:45.388486 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 15 10:18:45.388486 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:18:45.388486 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:18:45.388486 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:18:45.388486 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:18:45.388486 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:18:45.388486 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:18:45.388486 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:18:45.388486 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application with Lock( Jun 15 10:18:45.388486 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:18:45.388486 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:18:45.388486 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:18:45.388486 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:18:45.388486 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:18:45.388486 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:18:45.388486 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:18:45.388486 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:18:45.389775 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 15 10:18:45.389775 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 15 10:18:45.389775 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 15 10:18:45.389775 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 15 10:18:45.389775 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 15 10:18:45.389775 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 15 10:18:45.389775 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 10:18:45.389775 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application Jun 15 10:18:45.393046 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 265/534] 10.222.0.49 () {56 vars in 1111 bytes} [Wed Jun 15 10:18:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 320 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:18:45.398689 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ef80be-87e8-42c2-8e5a-881985edc729 None admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:18:45.406355 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc635c8-df8e-4c46-8df7-9239084be354 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:18:45.409080 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc635c8-df8e-4c46-8df7-9239084be354 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:18:45.441653 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application [None req-7bc635c8-df8e-4c46-8df7-9239084be354 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 10:18:45.441653 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:18:45.441653 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:18:45.441653 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:18:45.441653 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:18:45.441653 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:18:45.441653 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:18:45.441653 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:18:45.441653 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:18:45.441653 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:18:45.441653 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:18:45.441653 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:18:45.441653 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 10:18:45.441653 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 10:18:45.441653 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 10:18:45.442529 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 10:18:45.442529 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:18:45.442529 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:18:45.442529 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:18:45.442529 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:18:45.442529 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:18:45.442529 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:18:45.442529 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:18:45.442529 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application with Lock( Jun 15 10:18:45.442529 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:18:45.442529 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:18:45.442529 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:18:45.442529 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:18:45.442529 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:18:45.442529 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:18:45.442529 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:18:45.442529 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:18:45.443431 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 10:18:45.443431 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 10:18:45.443431 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 10:18:45.443431 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Jun 15 10:18:45.457250 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 270/535] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Jun 15 10:18:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:18:45.461029 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc635c8-df8e-4c46-8df7-9239084be354 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:18:45.540437 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:18:45.559253 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ef80be-87e8-42c2-8e5a-881985edc729 None admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:18:45.581524 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=100 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:18:45.603186 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86cbb0cf-b86f-43e4-8c93-43f74647ace0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:18:45.604551 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cbb0cf-b86f-43e4-8c93-43f74647ace0 None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:45.605079 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cbb0cf-b86f-43e4-8c93-43f74647ace0 None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:45.605564 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cbb0cf-b86f-43e4-8c93-43f74647ace0 None admin] PATH_INFO: `/v3/projects` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:45.606578 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86cbb0cf-b86f-43e4-8c93-43f74647ace0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:18:45.607120 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86cbb0cf-b86f-43e4-8c93-43f74647ace0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:18:45.609322 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86cbb0cf-b86f-43e4-8c93-43f74647ace0 None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:18:45.630687 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 266/536] 10.222.0.49 () {56 vars in 1148 bytes} [Wed Jun 15 10:18:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 10:18:45.637433 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86cbb0cf-b86f-43e4-8c93-43f74647ace0 None admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:18:45.737332 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc635c8-df8e-4c46-8df7-9239084be354 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:18:45.751946 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86cbb0cf-b86f-43e4-8c93-43f74647ace0 None admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:18:45.796862 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7a58a6-21aa-4486-8971-e3ab025a43ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:18:45.797536 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7a58a6-21aa-4486-8971-e3ab025a43ee None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:45.797826 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7a58a6-21aa-4486-8971-e3ab025a43ee None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:45.798107 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7a58a6-21aa-4486-8971-e3ab025a43ee None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:45.798941 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b7a58a6-21aa-4486-8971-e3ab025a43ee None admin] RBAC: Adding query filter params (role.id=16e76452bb5d4ce68a119ad3c9bde41c, scope.project.id=559b3e713f5844b6ae37151d56dd419f, user.id=20d1327553404b80aecee87c78fcb52b) {{(pid=80119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:18:45.799285 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b7a58a6-21aa-4486-8971-e3ab025a43ee None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:18:45.800737 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b7a58a6-21aa-4486-8971-e3ab025a43ee None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:18:45.807508 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 267/537] 10.222.0.49 () {56 vars in 1374 bytes} [Wed Jun 15 10:18:45 2022] GET /identity/v3/role_assignments?user.id=20d1327553404b80aecee87c78fcb52b&scope.project.id=559b3e713f5844b6ae37151d56dd419f&role.id=16e76452bb5d4ce68a119ad3c9bde41c => generated 259 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:18:45.818661 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db9a6c7-b5e7-4745-857e-aa58c4658bcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:18:45.819498 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db9a6c7-b5e7-4745-857e-aa58c4658bcd None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:45.819850 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db9a6c7-b5e7-4745-857e-aa58c4658bcd None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:45.820194 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db9a6c7-b5e7-4745-857e-aa58c4658bcd None admin] PATH_INFO: `/v3/domains` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:45.821002 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db9a6c7-b5e7-4745-857e-aa58c4658bcd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:18:45.821379 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db9a6c7-b5e7-4745-857e-aa58c4658bcd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:18:45.822975 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db9a6c7-b5e7-4745-857e-aa58c4658bcd None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:18:45.836955 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 271/538] 10.222.0.49 () {56 vars in 1109 bytes} [Wed Jun 15 10:18:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 376 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:18:45.844277 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7a58a6-21aa-4486-8971-e3ab025a43ee None admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:18:45.904542 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=100 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:18:45.969185 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7a58a6-21aa-4486-8971-e3ab025a43ee None admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:18:46.007014 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f9c07d-9fb3-4f31-b480-0161db5d10ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:18:46.007787 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f9c07d-9fb3-4f31-b480-0161db5d10ee None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:46.008069 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f9c07d-9fb3-4f31-b480-0161db5d10ee None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:46.008343 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f9c07d-9fb3-4f31-b480-0161db5d10ee None admin] PATH_INFO: `/v3/projects/service` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:46.016238 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f9c07d-9fb3-4f31-b480-0161db5d10ee None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:18:46.017816 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f9c07d-9fb3-4f31-b480-0161db5d10ee None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:18:46.032341 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application [None req-22f9c07d-9fb3-4f31-b480-0161db5d10ee None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 10:18:46.032341 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:18:46.032341 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:18:46.032341 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:18:46.032341 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:18:46.032341 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:18:46.032341 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:18:46.032341 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:18:46.032341 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:18:46.032341 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:18:46.032341 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:18:46.032341 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:18:46.032341 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 15 10:18:46.032341 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 15 10:18:46.032341 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 15 10:18:46.033201 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 15 10:18:46.033201 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:18:46.033201 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:18:46.033201 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:18:46.033201 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:18:46.033201 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:18:46.033201 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:18:46.033201 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:18:46.033201 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application with Lock( Jun 15 10:18:46.033201 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:18:46.033201 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:18:46.033201 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:18:46.033201 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:18:46.033201 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:18:46.033201 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:18:46.033201 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:18:46.033201 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:18:46.034071 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 15 10:18:46.034071 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 15 10:18:46.034071 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 15 10:18:46.034071 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 15 10:18:46.034071 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 15 10:18:46.034071 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 15 10:18:46.034071 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 10:18:46.034071 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application Jun 15 10:18:46.034071 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 268/539] 10.222.0.49 () {56 vars in 1111 bytes} [Wed Jun 15 10:18:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:18:46.040807 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db9a6c7-b5e7-4745-857e-aa58c4658bcd None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:18:46.155209 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:18:46.164428 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db9a6c7-b5e7-4745-857e-aa58c4658bcd None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:18:46.200896 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36322d7c-4b37-498d-bea3-c11cb28bf619 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:18:46.201569 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36322d7c-4b37-498d-bea3-c11cb28bf619 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:46.201853 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36322d7c-4b37-498d-bea3-c11cb28bf619 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:46.202131 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36322d7c-4b37-498d-bea3-c11cb28bf619 None admin] PATH_INFO: `/v3/projects` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:46.202919 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36322d7c-4b37-498d-bea3-c11cb28bf619 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:18:46.203326 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36322d7c-4b37-498d-bea3-c11cb28bf619 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:18:46.204845 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36322d7c-4b37-498d-bea3-c11cb28bf619 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:18:46.217939 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 272/540] 10.222.0.49 () {56 vars in 1148 bytes} [Wed Jun 15 10:18:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:18:46.223659 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f9c07d-9fb3-4f31-b480-0161db5d10ee None admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:18:46.337014 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f9c07d-9fb3-4f31-b480-0161db5d10ee None admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:18:46.338997 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=100 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:18:46.391792 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3156971e-cbfe-4a50-ae38-b13d0cb76800 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:18:46.392878 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3156971e-cbfe-4a50-ae38-b13d0cb76800 None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:46.393169 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3156971e-cbfe-4a50-ae38-b13d0cb76800 None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:46.393446 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3156971e-cbfe-4a50-ae38-b13d0cb76800 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:46.394283 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3156971e-cbfe-4a50-ae38-b13d0cb76800 None admin] RBAC: Adding query filter params (role.id=16e76452bb5d4ce68a119ad3c9bde41c, scope.project.id=559b3e713f5844b6ae37151d56dd419f, user.id=0cb44a17d205431793e9a8791e477a32) {{(pid=80119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:18:46.394595 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3156971e-cbfe-4a50-ae38-b13d0cb76800 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:18:46.396135 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3156971e-cbfe-4a50-ae38-b13d0cb76800 None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:18:46.402708 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 269/541] 10.222.0.49 () {56 vars in 1374 bytes} [Wed Jun 15 10:18:46 2022] GET /identity/v3/role_assignments?user.id=0cb44a17d205431793e9a8791e477a32&scope.project.id=559b3e713f5844b6ae37151d56dd419f&role.id=16e76452bb5d4ce68a119ad3c9bde41c => generated 259 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:18:46.582526 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3434cf52-9f99-4b05-bbb4-a581ccf7ee9a None None] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:46.583044 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3434cf52-9f99-4b05-bbb4-a581ccf7ee9a None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:46.583191 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3434cf52-9f99-4b05-bbb4-a581ccf7ee9a None None] PATH_INFO: `/` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:46.584910 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 273/542] 10.222.0.49 () {54 vars in 926 bytes} [Wed Jun 15 10:18:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:18:46.593445 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2c771b-fb1f-4958-8f14-b146e78319b9 None None] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:46.593718 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2c771b-fb1f-4958-8f14-b146e78319b9 None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:46.594000 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2c771b-fb1f-4958-8f14-b146e78319b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:46.615644 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.auth.core [None req-6d2c771b-fb1f-4958-8f14-b146e78319b9 None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:18:46.616902 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-6d2c771b-fb1f-4958-8f14-b146e78319b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:18:46.709800 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 270/543] 10.222.0.49 () {58 vars in 1023 bytes} [Wed Jun 15 10:18:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 10:18:46.728964 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883d376e-778c-4f92-924f-a3166b41a35c None None] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:46.729347 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883d376e-778c-4f92-924f-a3166b41a35c None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:46.729701 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883d376e-778c-4f92-924f-a3166b41a35c None None] PATH_INFO: `/` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:46.731455 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 274/544] 10.222.0.49 () {54 vars in 926 bytes} [Wed Jun 15 10:18:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:18:46.736916 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2c1cd3-2de3-46fa-bf31-dde2ab742457 None None] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:46.737887 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883d376e-778c-4f92-924f-a3166b41a35c None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:18:46.738886 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2c1cd3-2de3-46fa-bf31-dde2ab742457 None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:46.739207 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2c1cd3-2de3-46fa-bf31-dde2ab742457 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:46.768132 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.auth.core [None req-5b2c1cd3-2de3-46fa-bf31-dde2ab742457 None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:18:46.769361 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-5b2c1cd3-2de3-46fa-bf31-dde2ab742457 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:18:46.845453 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 271/545] 10.222.0.49 () {58 vars in 1023 bytes} [Wed Jun 15 10:18:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 10:18:46.867572 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2c1cd3-2de3-46fa-bf31-dde2ab742457 None None] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:18:46.899453 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=100 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:18:46.971335 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2c1cd3-2de3-46fa-bf31-dde2ab742457 None None] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:18:46.983751 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883d376e-778c-4f92-924f-a3166b41a35c None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:18:47.006510 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab6c1b1-9567-4fa6-8540-fc94f97e720f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:18:47.007347 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab6c1b1-9567-4fa6-8540-fc94f97e720f None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:47.007641 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab6c1b1-9567-4fa6-8540-fc94f97e720f None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:47.007920 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab6c1b1-9567-4fa6-8540-fc94f97e720f None admin] PATH_INFO: `/v3/roles/service` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:47.019787 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab6c1b1-9567-4fa6-8540-fc94f97e720f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 10:18:47.029037 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application [None req-8ab6c1b1-9567-4fa6-8540-fc94f97e720f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 15 10:18:47.029037 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:18:47.029037 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:18:47.029037 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:18:47.029037 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:18:47.029037 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:18:47.029037 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:18:47.029037 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:18:47.029037 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:18:47.029037 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:18:47.029037 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:18:47.029037 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:18:47.029037 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 10:18:47.029037 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 10:18:47.029037 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 10:18:47.029037 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 10:18:47.029954 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 10:18:47.029954 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 10:18:47.029954 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 10:18:47.029954 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 10:18:47.029954 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:18:47.029954 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:18:47.029954 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:18:47.029954 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:18:47.029954 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:18:47.029954 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:18:47.029954 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:18:47.029954 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application with Lock( Jun 15 10:18:47.029954 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:18:47.029954 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:18:47.029954 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:18:47.029954 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:18:47.030731 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:18:47.030731 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:18:47.030731 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:18:47.030731 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:18:47.030731 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 10:18:47.030731 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 10:18:47.030731 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 10:18:47.030731 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 10:18:47.030731 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 10:18:47.030731 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 10:18:47.030731 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 15 10:18:47.030731 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application Jun 15 10:18:47.031379 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 272/546] 10.222.0.49 () {56 vars in 1102 bytes} [Wed Jun 15 10:18:46 2022] GET /identity/v3/roles/service => generated 85 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 10:18:47.037004 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab6c1b1-9567-4fa6-8540-fc94f97e720f None admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:18:47.115884 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8787afde-e4e4-46b6-acfd-0aa20e54fdf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:18:47.116638 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8787afde-e4e4-46b6-acfd-0aa20e54fdf7 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:47.116928 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8787afde-e4e4-46b6-acfd-0aa20e54fdf7 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:47.117209 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8787afde-e4e4-46b6-acfd-0aa20e54fdf7 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:47.127494 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=100 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:18:47.145623 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab6c1b1-9567-4fa6-8540-fc94f97e720f None admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:18:47.168074 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8787afde-e4e4-46b6-acfd-0aa20e54fdf7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 10:18:47.180007 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application [None req-8787afde-e4e4-46b6-acfd-0aa20e54fdf7 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 15 10:18:47.180007 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:18:47.180007 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:18:47.180007 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:18:47.180007 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:18:47.180007 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:18:47.180007 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:18:47.180007 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:18:47.180007 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:18:47.180007 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:18:47.180007 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:18:47.180007 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:18:47.180007 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 10:18:47.180007 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 10:18:47.180007 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 10:18:47.180007 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 10:18:47.182154 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 10:18:47.182154 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 10:18:47.182154 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 10:18:47.182154 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 10:18:47.182154 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:18:47.182154 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:18:47.182154 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:18:47.182154 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:18:47.182154 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:18:47.182154 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:18:47.182154 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:18:47.182154 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application with Lock( Jun 15 10:18:47.182154 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:18:47.182154 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:18:47.182154 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:18:47.182154 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:18:47.185017 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:18:47.185017 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:18:47.185017 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:18:47.185017 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:18:47.185017 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 10:18:47.185017 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 10:18:47.185017 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 10:18:47.185017 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 10:18:47.185017 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 10:18:47.185017 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 10:18:47.185017 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 15 10:18:47.185017 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Jun 15 10:18:47.185017 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 275/547] 10.222.0.49 () {56 vars in 1102 bytes} [Wed Jun 15 10:18:46 2022] GET /identity/v3/roles/service => generated 85 bytes in 444 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 10:18:47.186628 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e88e2f-1d1b-4063-b97c-43d61e6c7fd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:18:47.186628 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e88e2f-1d1b-4063-b97c-43d61e6c7fd6 None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:47.186628 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e88e2f-1d1b-4063-b97c-43d61e6c7fd6 None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:47.186628 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e88e2f-1d1b-4063-b97c-43d61e6c7fd6 None admin] PATH_INFO: `/v3/roles` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:47.192748 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e88e2f-1d1b-4063-b97c-43d61e6c7fd6 None admin] RBAC: Adding query filter params (name=service) {{(pid=80119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:18:47.192748 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e88e2f-1d1b-4063-b97c-43d61e6c7fd6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:18:47.192748 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e88e2f-1d1b-4063-b97c-43d61e6c7fd6 None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:18:47.193347 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8787afde-e4e4-46b6-acfd-0aa20e54fdf7 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:18:47.215615 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 273/548] 10.222.0.49 () {56 vars in 1103 bytes} [Wed Jun 15 10:18:47 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 10:18:47.217718 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e88e2f-1d1b-4063-b97c-43d61e6c7fd6 None admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:18:47.220013 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:18:47.385184 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8787afde-e4e4-46b6-acfd-0aa20e54fdf7 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:18:47.406869 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e88e2f-1d1b-4063-b97c-43d61e6c7fd6 None admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:18:47.449079 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1420884a-7ceb-47b1-be76-9929a86bde47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:18:47.450202 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1420884a-7ceb-47b1-be76-9929a86bde47 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:47.450684 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1420884a-7ceb-47b1-be76-9929a86bde47 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:47.451188 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1420884a-7ceb-47b1-be76-9929a86bde47 None admin] PATH_INFO: `/v3/roles` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:47.452136 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1420884a-7ceb-47b1-be76-9929a86bde47 None admin] RBAC: Adding query filter params (name=service) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:18:47.453674 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1420884a-7ceb-47b1-be76-9929a86bde47 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:18:47.455291 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1420884a-7ceb-47b1-be76-9929a86bde47 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:18:47.472939 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 276/549] 10.222.0.49 () {56 vars in 1103 bytes} [Wed Jun 15 10:18:47 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 10:18:47.479057 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1420884a-7ceb-47b1-be76-9929a86bde47 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:18:47.485657 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213a4c22-76c5-4299-b3ff-d8c9f7c48731 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:18:47.486306 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213a4c22-76c5-4299-b3ff-d8c9f7c48731 None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:47.486600 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213a4c22-76c5-4299-b3ff-d8c9f7c48731 None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:47.486879 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213a4c22-76c5-4299-b3ff-d8c9f7c48731 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:47.507925 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213a4c22-76c5-4299-b3ff-d8c9f7c48731 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:18:47.509399 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213a4c22-76c5-4299-b3ff-d8c9f7c48731 None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:18:47.527154 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=100 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:18:47.546040 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application [None req-213a4c22-76c5-4299-b3ff-d8c9f7c48731 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 10:18:47.546040 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:18:47.546040 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:18:47.546040 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:18:47.546040 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:18:47.546040 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:18:47.546040 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:18:47.546040 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:18:47.546040 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:18:47.546040 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:18:47.546040 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:18:47.546040 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:18:47.546040 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 10:18:47.546040 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 10:18:47.546040 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 10:18:47.547451 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 10:18:47.547451 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:18:47.547451 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:18:47.547451 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:18:47.547451 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:18:47.547451 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:18:47.547451 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:18:47.547451 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:18:47.547451 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application with Lock( Jun 15 10:18:47.547451 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:18:47.547451 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:18:47.547451 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:18:47.547451 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:18:47.547451 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:18:47.547451 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:18:47.547451 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:18:47.547451 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:18:47.548477 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 10:18:47.548477 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 10:18:47.548477 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 10:18:47.548477 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application Jun 15 10:18:47.548477 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 274/550] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Jun 15 10:18:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 330 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:18:47.555181 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213a4c22-76c5-4299-b3ff-d8c9f7c48731 None admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:18:47.629751 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:18:47.694209 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213a4c22-76c5-4299-b3ff-d8c9f7c48731 None admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:18:47.713106 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1420884a-7ceb-47b1-be76-9929a86bde47 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:18:47.735662 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ebd8d1-d999-4555-8f58-70979ffd11d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:18:47.736729 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ebd8d1-d999-4555-8f58-70979ffd11d3 None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:47.737212 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ebd8d1-d999-4555-8f58-70979ffd11d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:47.737700 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ebd8d1-d999-4555-8f58-70979ffd11d3 None admin] PATH_INFO: `/v3/domains` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:47.738936 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ebd8d1-d999-4555-8f58-70979ffd11d3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:18:47.740489 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ebd8d1-d999-4555-8f58-70979ffd11d3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:18:47.743950 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ebd8d1-d999-4555-8f58-70979ffd11d3 None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:18:47.767423 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 275/551] 10.222.0.49 () {56 vars in 1109 bytes} [Wed Jun 15 10:18:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:18:47.775634 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ebd8d1-d999-4555-8f58-70979ffd11d3 None admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:18:47.797058 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a449f9-3028-4903-a171-1efdb3f55519 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:18:47.797878 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a449f9-3028-4903-a171-1efdb3f55519 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:47.798161 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a449f9-3028-4903-a171-1efdb3f55519 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:47.798437 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a449f9-3028-4903-a171-1efdb3f55519 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:47.824265 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a449f9-3028-4903-a171-1efdb3f55519 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:18:47.826054 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a449f9-3028-4903-a171-1efdb3f55519 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:18:47.854081 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application [None req-d8a449f9-3028-4903-a171-1efdb3f55519 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 10:18:47.854081 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:18:47.854081 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:18:47.854081 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:18:47.854081 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:18:47.854081 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:18:47.854081 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:18:47.854081 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:18:47.854081 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:18:47.854081 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:18:47.854081 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:18:47.854081 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:18:47.854081 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 10:18:47.854081 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 10:18:47.854081 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 10:18:47.855751 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 10:18:47.855751 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:18:47.855751 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:18:47.855751 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:18:47.855751 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:18:47.855751 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:18:47.855751 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:18:47.855751 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:18:47.855751 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application with Lock( Jun 15 10:18:47.855751 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:18:47.855751 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:18:47.855751 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:18:47.855751 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:18:47.855751 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:18:47.855751 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:18:47.855751 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:18:47.855751 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:18:47.856797 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 10:18:47.856797 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 10:18:47.856797 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 10:18:47.856797 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Jun 15 10:18:47.869100 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 277/552] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Jun 15 10:18:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 390 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:18:47.871460 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=100 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:18:47.873328 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a449f9-3028-4903-a171-1efdb3f55519 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:18:47.925110 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ebd8d1-d999-4555-8f58-70979ffd11d3 None admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:18:47.966696 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ea6b4a-5f68-4eab-ada1-b645c0a4e13d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:18:47.967467 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ea6b4a-5f68-4eab-ada1-b645c0a4e13d None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:47.967756 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ea6b4a-5f68-4eab-ada1-b645c0a4e13d None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:47.968038 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ea6b4a-5f68-4eab-ada1-b645c0a4e13d None admin] PATH_INFO: `/v3/users/nova` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:47.976592 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ea6b4a-5f68-4eab-ada1-b645c0a4e13d None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:18:47.978057 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ea6b4a-5f68-4eab-ada1-b645c0a4e13d None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:18:47.992629 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application [None req-f8ea6b4a-5f68-4eab-ada1-b645c0a4e13d None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 15 10:18:47.992629 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:18:47.992629 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:18:47.992629 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:18:47.992629 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:18:47.992629 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:18:47.992629 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:18:47.992629 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:18:47.992629 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:18:47.992629 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:18:47.992629 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:18:47.992629 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:18:47.992629 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 15 10:18:47.992629 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 15 10:18:47.992629 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 15 10:18:47.992629 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 15 10:18:47.993619 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:18:47.993619 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:18:47.993619 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 15 10:18:47.993619 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 10:18:47.993619 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 15 10:18:47.993619 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 10:18:47.993619 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:18:47.993619 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:18:47.993619 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:18:47.993619 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:18:47.993619 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:18:47.993619 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application with Lock( Jun 15 10:18:47.993619 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:18:47.993619 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:18:47.993619 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:18:47.993619 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:18:47.993619 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:18:47.993619 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:18:47.995198 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:18:47.995198 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:18:47.995198 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 15 10:18:47.995198 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 15 10:18:47.995198 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 15 10:18:47.995198 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 15 10:18:47.995198 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 15 10:18:47.995198 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 15 10:18:47.995198 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 15 10:18:47.995198 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application Jun 15 10:18:47.995198 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 276/553] 10.222.0.49 () {56 vars in 1093 bytes} [Wed Jun 15 10:18:47 2022] GET /identity/v3/users/nova => generated 82 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:18:47.995886 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efadfcd-a3b5-4ffd-a445-2470a45b74fc None None] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:47.995886 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efadfcd-a3b5-4ffd-a445-2470a45b74fc None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:47.996217 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efadfcd-a3b5-4ffd-a445-2470a45b74fc None None] PATH_INFO: `/` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:47.997577 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 277/554] 10.222.0.49 () {54 vars in 926 bytes} [Wed Jun 15 10:18:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:18:48.002865 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e979038-15e4-49d2-a87d-ba2a330d82d2 None None] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:48.003218 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e979038-15e4-49d2-a87d-ba2a330d82d2 None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:48.003511 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e979038-15e4-49d2-a87d-ba2a330d82d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:48.020178 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.auth.core [None req-6e979038-15e4-49d2-a87d-ba2a330d82d2 None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:18:48.021414 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-6e979038-15e4-49d2-a87d-ba2a330d82d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:18:48.051135 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a449f9-3028-4903-a171-1efdb3f55519 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:18:48.078323 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:18:48.088365 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df658e76-a7af-4127-bfae-bcb4a4468b21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:18:48.090175 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df658e76-a7af-4127-bfae-bcb4a4468b21 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:48.090661 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df658e76-a7af-4127-bfae-bcb4a4468b21 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:48.091178 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df658e76-a7af-4127-bfae-bcb4a4468b21 None admin] PATH_INFO: `/v3/domains` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:48.092459 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df658e76-a7af-4127-bfae-bcb4a4468b21 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:18:48.093240 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df658e76-a7af-4127-bfae-bcb4a4468b21 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:18:48.094681 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 278/555] 10.222.0.49 () {58 vars in 1023 bytes} [Wed Jun 15 10:18:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 10:18:48.095588 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e979038-15e4-49d2-a87d-ba2a330d82d2 None None] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:18:48.095877 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df658e76-a7af-4127-bfae-bcb4a4468b21 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:18:48.110474 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 278/556] 10.222.0.49 () {56 vars in 1109 bytes} [Wed Jun 15 10:18:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:18:48.116402 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df658e76-a7af-4127-bfae-bcb4a4468b21 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:18:48.173622 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=100 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:18:48.231052 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e979038-15e4-49d2-a87d-ba2a330d82d2 None None] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:18:48.246743 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df658e76-a7af-4127-bfae-bcb4a4468b21 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:18:48.266609 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c0c845-17ee-4e61-98f3-a40fc46b7926 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:18:48.267319 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c0c845-17ee-4e61-98f3-a40fc46b7926 None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:48.267611 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c0c845-17ee-4e61-98f3-a40fc46b7926 None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:48.267893 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c0c845-17ee-4e61-98f3-a40fc46b7926 None admin] PATH_INFO: `/v3/users` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:48.268737 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c0c845-17ee-4e61-98f3-a40fc46b7926 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:18:48.269046 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c0c845-17ee-4e61-98f3-a40fc46b7926 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:18:48.270493 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c0c845-17ee-4e61-98f3-a40fc46b7926 None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:18:48.283737 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b485cc3-d185-4030-bd73-bede6f9032d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:18:48.284536 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b485cc3-d185-4030-bd73-bede6f9032d0 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:48.284820 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b485cc3-d185-4030-bd73-bede6f9032d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:48.285094 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b485cc3-d185-4030-bd73-bede6f9032d0 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:48.291030 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 279/557] 10.222.0.49 () {56 vars in 1133 bytes} [Wed Jun 15 10:18:48 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:18:48.292088 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c0c845-17ee-4e61-98f3-a40fc46b7926 None admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:18:48.293030 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b485cc3-d185-4030-bd73-bede6f9032d0 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:18:48.294490 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b485cc3-d185-4030-bd73-bede6f9032d0 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:18:48.312783 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application [None req-1b485cc3-d185-4030-bd73-bede6f9032d0 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 15 10:18:48.312783 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:18:48.312783 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:18:48.312783 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:18:48.312783 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:18:48.312783 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:18:48.312783 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:18:48.312783 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:18:48.312783 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:18:48.312783 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:18:48.312783 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:18:48.312783 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:18:48.312783 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 15 10:18:48.312783 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 15 10:18:48.312783 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 15 10:18:48.312783 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 15 10:18:48.314520 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:18:48.314520 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:18:48.314520 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 15 10:18:48.314520 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 10:18:48.314520 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 15 10:18:48.314520 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 10:18:48.314520 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:18:48.314520 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:18:48.314520 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:18:48.314520 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:18:48.314520 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:18:48.314520 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application with Lock( Jun 15 10:18:48.314520 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:18:48.314520 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:18:48.314520 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:18:48.314520 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:18:48.314520 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:18:48.314520 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:18:48.315542 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:18:48.315542 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:18:48.315542 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 15 10:18:48.315542 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 15 10:18:48.315542 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 15 10:18:48.315542 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 15 10:18:48.315542 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 15 10:18:48.315542 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 15 10:18:48.315542 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 15 10:18:48.315542 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Jun 15 10:18:48.315542 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 279/558] 10.222.0.49 () {56 vars in 1096 bytes} [Wed Jun 15 10:18:48 2022] GET /identity/v3/users/swift => generated 83 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 10:18:48.315542 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67327a45-2e94-41f0-8004-182b4313cc78 None None] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:48.315542 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67327a45-2e94-41f0-8004-182b4313cc78 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:48.316252 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67327a45-2e94-41f0-8004-182b4313cc78 None None] PATH_INFO: `/` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:48.320964 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 280/559] 10.222.0.49 () {54 vars in 926 bytes} [Wed Jun 15 10:18:48 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:18:48.321740 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67327a45-2e94-41f0-8004-182b4313cc78 None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:18:48.374579 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:18:48.400767 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c0c845-17ee-4e61-98f3-a40fc46b7926 None admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:18:48.411233 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=100 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:18:48.435151 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67327a45-2e94-41f0-8004-182b4313cc78 None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:18:48.438694 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5623a9d9-1634-4d61-b8b1-f7be1fc3856a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:18:48.439524 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5623a9d9-1634-4d61-b8b1-f7be1fc3856a None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:48.439810 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5623a9d9-1634-4d61-b8b1-f7be1fc3856a None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:48.440084 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5623a9d9-1634-4d61-b8b1-f7be1fc3856a None admin] PATH_INFO: `/v3/roles/service` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:48.451292 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5623a9d9-1634-4d61-b8b1-f7be1fc3856a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 10:18:48.458985 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application [None req-5623a9d9-1634-4d61-b8b1-f7be1fc3856a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 15 10:18:48.458985 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:18:48.458985 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:18:48.458985 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:18:48.458985 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:18:48.458985 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:18:48.458985 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:18:48.458985 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:18:48.458985 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:18:48.458985 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:18:48.458985 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:18:48.458985 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:18:48.458985 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 10:18:48.458985 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 10:18:48.458985 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 10:18:48.458985 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 10:18:48.460267 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 10:18:48.460267 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 10:18:48.460267 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 10:18:48.460267 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 10:18:48.460267 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:18:48.460267 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:18:48.460267 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:18:48.460267 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:18:48.460267 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:18:48.460267 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:18:48.460267 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:18:48.460267 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application with Lock( Jun 15 10:18:48.460267 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:18:48.460267 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:18:48.460267 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:18:48.460267 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:18:48.461311 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:18:48.461311 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:18:48.461311 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:18:48.461311 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:18:48.461311 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 10:18:48.461311 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 10:18:48.461311 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 10:18:48.461311 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 10:18:48.461311 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 10:18:48.461311 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 10:18:48.461311 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 15 10:18:48.461311 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application Jun 15 10:18:48.461311 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 280/560] 10.222.0.49 () {56 vars in 1102 bytes} [Wed Jun 15 10:18:48 2022] GET /identity/v3/roles/service => generated 85 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:18:48.462181 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5623a9d9-1634-4d61-b8b1-f7be1fc3856a None admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:18:48.481461 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2ef807-818b-4b09-977b-75c735217ccc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:18:48.482461 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2ef807-818b-4b09-977b-75c735217ccc None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:48.482752 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2ef807-818b-4b09-977b-75c735217ccc None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:48.483312 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2ef807-818b-4b09-977b-75c735217ccc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:48.494729 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2ef807-818b-4b09-977b-75c735217ccc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:18:48.496271 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2ef807-818b-4b09-977b-75c735217ccc None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:18:48.517214 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application [None req-1b2ef807-818b-4b09-977b-75c735217ccc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 10:18:48.517214 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:18:48.517214 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:18:48.517214 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:18:48.517214 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:18:48.517214 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:18:48.517214 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:18:48.517214 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:18:48.517214 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:18:48.517214 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:18:48.517214 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:18:48.517214 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:18:48.517214 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 10:18:48.517214 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 10:18:48.517214 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 10:18:48.518157 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 10:18:48.518157 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:18:48.518157 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:18:48.518157 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:18:48.518157 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:18:48.518157 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:18:48.518157 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:18:48.518157 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:18:48.518157 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application with Lock( Jun 15 10:18:48.518157 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:18:48.518157 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:18:48.518157 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:18:48.518157 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:18:48.518157 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:18:48.518157 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:18:48.518157 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:18:48.518157 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:18:48.519040 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 10:18:48.519040 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 10:18:48.519040 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 10:18:48.519040 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Jun 15 10:18:48.519040 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 281/561] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Jun 15 10:18:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:18:48.519889 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748437b1-6301-45fa-be32-3d044441d9b5 None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:48.519990 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748437b1-6301-45fa-be32-3d044441d9b5 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:48.520298 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748437b1-6301-45fa-be32-3d044441d9b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:48.538532 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-748437b1-6301-45fa-be32-3d044441d9b5 None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:18:48.539771 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-748437b1-6301-45fa-be32-3d044441d9b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:18:48.575327 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5623a9d9-1634-4d61-b8b1-f7be1fc3856a None admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:18:48.610237 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ca2716-2168-4e16-84a2-329ab7a965ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:18:48.610940 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ca2716-2168-4e16-84a2-329ab7a965ce None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:48.611263 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ca2716-2168-4e16-84a2-329ab7a965ce None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:48.611542 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ca2716-2168-4e16-84a2-329ab7a965ce None admin] PATH_INFO: `/v3/users` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:48.612374 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ca2716-2168-4e16-84a2-329ab7a965ce None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:18:48.612682 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ca2716-2168-4e16-84a2-329ab7a965ce None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:18:48.614154 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ca2716-2168-4e16-84a2-329ab7a965ce None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:18:48.620169 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 282/562] 10.222.0.49 () {58 vars in 1023 bytes} [Wed Jun 15 10:18:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 10:18:48.622032 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748437b1-6301-45fa-be32-3d044441d9b5 None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:18:48.637328 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 281/563] 10.222.0.49 () {56 vars in 1135 bytes} [Wed Jun 15 10:18:48 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:18:48.638227 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ca2716-2168-4e16-84a2-329ab7a965ce None admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:18:48.661826 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:18:48.667047 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=100 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:18:48.744811 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ca2716-2168-4e16-84a2-329ab7a965ce None admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:18:48.745382 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748437b1-6301-45fa-be32-3d044441d9b5 None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:18:48.782442 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a08913-ed2a-468b-8676-68cc1b43168b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:18:48.782875 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bfe8223-37d4-4d27-896d-bbe8586d945e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:18:48.783149 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a08913-ed2a-468b-8676-68cc1b43168b None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:48.783442 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a08913-ed2a-468b-8676-68cc1b43168b None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:48.783582 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfe8223-37d4-4d27-896d-bbe8586d945e None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:48.783722 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a08913-ed2a-468b-8676-68cc1b43168b None admin] PATH_INFO: `/v3/domains` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:48.783866 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfe8223-37d4-4d27-896d-bbe8586d945e None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:48.784140 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfe8223-37d4-4d27-896d-bbe8586d945e None admin] PATH_INFO: `/v3/roles` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:48.784444 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a08913-ed2a-468b-8676-68cc1b43168b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:18:48.784754 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a08913-ed2a-468b-8676-68cc1b43168b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:18:48.784886 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bfe8223-37d4-4d27-896d-bbe8586d945e None admin] RBAC: Adding query filter params (name=service) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:18:48.785190 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bfe8223-37d4-4d27-896d-bbe8586d945e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:18:48.786145 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a08913-ed2a-468b-8676-68cc1b43168b None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:18:48.786547 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bfe8223-37d4-4d27-896d-bbe8586d945e None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:18:48.794925 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 283/564] 10.222.0.49 () {56 vars in 1103 bytes} [Wed Jun 15 10:18:48 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:18:48.795897 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bfe8223-37d4-4d27-896d-bbe8586d945e None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:18:48.802373 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 282/565] 10.222.0.49 () {56 vars in 1109 bytes} [Wed Jun 15 10:18:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:18:48.802373 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a08913-ed2a-468b-8676-68cc1b43168b None admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:18:48.883211 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=100 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:18:48.891436 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:18:48.900250 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a08913-ed2a-468b-8676-68cc1b43168b None admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:18:48.901136 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bfe8223-37d4-4d27-896d-bbe8586d945e None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:18:48.934281 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80262f2-10db-42cf-8ac9-c9c127e911d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:18:48.934932 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80262f2-10db-42cf-8ac9-c9c127e911d0 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:48.935245 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80262f2-10db-42cf-8ac9-c9c127e911d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:48.935520 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80262f2-10db-42cf-8ac9-c9c127e911d0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:48.936298 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477d82bb-ebab-4a8a-b80c-04a9483f548f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:18:48.937094 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477d82bb-ebab-4a8a-b80c-04a9483f548f None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:48.937381 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477d82bb-ebab-4a8a-b80c-04a9483f548f None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:48.937662 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477d82bb-ebab-4a8a-b80c-04a9483f548f None admin] PATH_INFO: `/v3/roles/service` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:48.947646 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-477d82bb-ebab-4a8a-b80c-04a9483f548f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 10:18:48.955353 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application [None req-477d82bb-ebab-4a8a-b80c-04a9483f548f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 15 10:18:48.955353 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:18:48.955353 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:18:48.955353 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:18:48.955353 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:18:48.955353 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:18:48.955353 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:18:48.955353 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:18:48.955353 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:18:48.955353 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:18:48.955353 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:18:48.955353 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:18:48.955353 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 10:18:48.955353 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 10:18:48.955353 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 10:18:48.955353 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 10:18:48.956267 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 10:18:48.956267 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 10:18:48.956267 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 10:18:48.956267 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 10:18:48.956267 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:18:48.956267 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:18:48.956267 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:18:48.956267 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:18:48.956267 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:18:48.956267 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:18:48.956267 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:18:48.956267 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application with Lock( Jun 15 10:18:48.956267 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:18:48.956267 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:18:48.956267 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:18:48.956267 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:18:48.957087 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:18:48.957087 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:18:48.957087 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:18:48.957087 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:18:48.957087 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 10:18:48.957087 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 10:18:48.957087 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 10:18:48.957087 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 10:18:48.957087 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 10:18:48.957087 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 10:18:48.957087 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 15 10:18:48.957087 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application Jun 15 10:18:48.957087 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 283/566] 10.222.0.49 () {56 vars in 1102 bytes} [Wed Jun 15 10:18:48 2022] GET /identity/v3/roles/service => generated 85 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:18:48.957777 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9acbce-9db2-4af5-96d1-2ed5acb018c2 None None] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:48.958038 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9acbce-9db2-4af5-96d1-2ed5acb018c2 None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:48.958299 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9acbce-9db2-4af5-96d1-2ed5acb018c2 None None] PATH_INFO: `/` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:48.959548 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 284/567] 10.222.0.49 () {54 vars in 926 bytes} [Wed Jun 15 10:18:48 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:18:48.960248 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9acbce-9db2-4af5-96d1-2ed5acb018c2 None None] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:18:49.008147 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80262f2-10db-42cf-8ac9-c9c127e911d0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:18:49.009541 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80262f2-10db-42cf-8ac9-c9c127e911d0 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:18:49.062738 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9acbce-9db2-4af5-96d1-2ed5acb018c2 None None] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:18:49.094864 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1536c7df-23b5-48c5-9323-431b038ce791 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:18:49.095677 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1536c7df-23b5-48c5-9323-431b038ce791 None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:49.095919 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:18:49.096016 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1536c7df-23b5-48c5-9323-431b038ce791 None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:49.096219 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1536c7df-23b5-48c5-9323-431b038ce791 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:49.107466 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 284/568] 10.222.0.49 () {58 vars in 1282 bytes} [Wed Jun 15 10:18:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 15 10:18:49.107604 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1536c7df-23b5-48c5-9323-431b038ce791 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:18:49.108276 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80262f2-10db-42cf-8ac9-c9c127e911d0 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:18:49.108931 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1536c7df-23b5-48c5-9323-431b038ce791 None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:18:49.117136 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=100 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:18:49.129527 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application [None req-1536c7df-23b5-48c5-9323-431b038ce791 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 10:18:49.129527 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:18:49.129527 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:18:49.129527 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:18:49.129527 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:18:49.129527 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:18:49.129527 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:18:49.129527 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:18:49.129527 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:18:49.129527 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:18:49.129527 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:18:49.129527 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:18:49.129527 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 10:18:49.129527 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 10:18:49.129527 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 10:18:49.130430 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 10:18:49.130430 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:18:49.130430 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:18:49.130430 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:18:49.130430 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:18:49.130430 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:18:49.130430 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:18:49.130430 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:18:49.130430 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application with Lock( Jun 15 10:18:49.130430 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:18:49.130430 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:18:49.130430 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:18:49.130430 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:18:49.130430 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:18:49.130430 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:18:49.130430 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:18:49.130430 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:18:49.131373 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 10:18:49.131373 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 10:18:49.131373 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 10:18:49.131373 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application Jun 15 10:18:49.131373 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 285/569] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Jun 15 10:18:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:18:49.131810 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1536c7df-23b5-48c5-9323-431b038ce791 None admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:18:49.206227 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80262f2-10db-42cf-8ac9-c9c127e911d0 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:18:49.230664 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1536c7df-23b5-48c5-9323-431b038ce791 None admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:18:49.239063 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb537b40-9e1c-41e1-999d-e443d280b5fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:18:49.239881 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb537b40-9e1c-41e1-999d-e443d280b5fc None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:49.240167 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb537b40-9e1c-41e1-999d-e443d280b5fc None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:49.240441 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb537b40-9e1c-41e1-999d-e443d280b5fc None admin] PATH_INFO: `/v3/projects/service` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:49.247542 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb537b40-9e1c-41e1-999d-e443d280b5fc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:18:49.248979 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb537b40-9e1c-41e1-999d-e443d280b5fc None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:18:49.263058 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application [None req-cb537b40-9e1c-41e1-999d-e443d280b5fc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 10:18:49.263058 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:18:49.263058 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:18:49.263058 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:18:49.263058 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:18:49.263058 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:18:49.263058 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:18:49.263058 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:18:49.263058 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:18:49.263058 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:18:49.263058 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:18:49.263058 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:18:49.263058 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 15 10:18:49.263058 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 15 10:18:49.263058 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 15 10:18:49.264007 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 15 10:18:49.264007 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:18:49.264007 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:18:49.264007 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:18:49.264007 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:18:49.264007 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:18:49.264007 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:18:49.264007 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:18:49.264007 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application with Lock( Jun 15 10:18:49.264007 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:18:49.264007 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:18:49.264007 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:18:49.264007 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:18:49.264007 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:18:49.264007 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:18:49.264007 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:18:49.264007 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:18:49.265120 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 15 10:18:49.265120 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 15 10:18:49.265120 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 15 10:18:49.265120 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 15 10:18:49.265120 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 15 10:18:49.265120 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 15 10:18:49.265120 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 10:18:49.265120 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Jun 15 10:18:49.265712 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b744d1ef-99a4-4bae-a320-a85a12eb0ba6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:18:49.265712 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b744d1ef-99a4-4bae-a320-a85a12eb0ba6 None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:49.265859 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 285/570] 10.222.0.49 () {56 vars in 1111 bytes} [Wed Jun 15 10:18:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:18:49.265936 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b744d1ef-99a4-4bae-a320-a85a12eb0ba6 None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:49.265936 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b744d1ef-99a4-4bae-a320-a85a12eb0ba6 None admin] PATH_INFO: `/v3/roles` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:49.265936 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b744d1ef-99a4-4bae-a320-a85a12eb0ba6 None admin] RBAC: Adding query filter params (name=service) {{(pid=80119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:18:49.266120 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8914607e-1f12-4d2d-ab82-4d1d9cfa5907 None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:49.266187 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b744d1ef-99a4-4bae-a320-a85a12eb0ba6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:18:49.266253 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8914607e-1f12-4d2d-ab82-4d1d9cfa5907 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:49.266253 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8914607e-1f12-4d2d-ab82-4d1d9cfa5907 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:49.267230 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b744d1ef-99a4-4bae-a320-a85a12eb0ba6 None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:18:49.275460 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 286/571] 10.222.0.49 () {56 vars in 1103 bytes} [Wed Jun 15 10:18:49 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:18:49.276247 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b744d1ef-99a4-4bae-a320-a85a12eb0ba6 None admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:18:49.284021 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-8914607e-1f12-4d2d-ab82-4d1d9cfa5907 None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:18:49.285217 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-8914607e-1f12-4d2d-ab82-4d1d9cfa5907 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:18:49.330349 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:18:49.335371 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=100 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:18:49.351973 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 286/572] 10.222.0.49 () {58 vars in 1023 bytes} [Wed Jun 15 10:18:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 10:18:49.353148 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8914607e-1f12-4d2d-ab82-4d1d9cfa5907 None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:18:49.377376 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b744d1ef-99a4-4bae-a320-a85a12eb0ba6 None admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:18:49.409849 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2117e79-15e9-4a97-9c09-87c74a5a1f35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:18:49.410635 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2117e79-15e9-4a97-9c09-87c74a5a1f35 None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:49.410916 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2117e79-15e9-4a97-9c09-87c74a5a1f35 None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:49.411225 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2117e79-15e9-4a97-9c09-87c74a5a1f35 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:49.422073 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2117e79-15e9-4a97-9c09-87c74a5a1f35 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:18:49.423579 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2117e79-15e9-4a97-9c09-87c74a5a1f35 None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:18:49.440462 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application [None req-b2117e79-15e9-4a97-9c09-87c74a5a1f35 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 10:18:49.440462 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:18:49.440462 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:18:49.440462 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:18:49.440462 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:18:49.440462 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:18:49.440462 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:18:49.440462 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:18:49.440462 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:18:49.440462 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:18:49.440462 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:18:49.440462 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:18:49.440462 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 10:18:49.440462 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 10:18:49.440462 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 10:18:49.441391 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 10:18:49.441391 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:18:49.441391 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:18:49.441391 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:18:49.441391 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:18:49.441391 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:18:49.441391 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:18:49.441391 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:18:49.441391 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application with Lock( Jun 15 10:18:49.441391 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:18:49.441391 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:18:49.441391 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:18:49.441391 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:18:49.441391 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:18:49.441391 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:18:49.441391 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:18:49.441391 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:18:49.442291 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 10:18:49.442291 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 10:18:49.442291 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 10:18:49.442291 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application Jun 15 10:18:49.442291 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 287/573] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Jun 15 10:18:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:18:49.442578 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2117e79-15e9-4a97-9c09-87c74a5a1f35 None admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:18:49.451535 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8914607e-1f12-4d2d-ab82-4d1d9cfa5907 None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:18:49.485510 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9612c960-bc14-4f3e-a5c6-9b56f05cf32e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:18:49.486176 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9612c960-bc14-4f3e-a5c6-9b56f05cf32e None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:49.486464 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9612c960-bc14-4f3e-a5c6-9b56f05cf32e None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:49.486750 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9612c960-bc14-4f3e-a5c6-9b56f05cf32e None admin] PATH_INFO: `/v3/domains` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:49.487509 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9612c960-bc14-4f3e-a5c6-9b56f05cf32e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:18:49.487819 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9612c960-bc14-4f3e-a5c6-9b56f05cf32e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:18:49.489187 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9612c960-bc14-4f3e-a5c6-9b56f05cf32e None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:18:49.500148 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 287/574] 10.222.0.49 () {56 vars in 1109 bytes} [Wed Jun 15 10:18:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:18:49.500926 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9612c960-bc14-4f3e-a5c6-9b56f05cf32e None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:18:49.543988 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2117e79-15e9-4a97-9c09-87c74a5a1f35 None admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:18:49.544711 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:18:49.557014 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=100 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:18:49.577686 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61468e64-0d51-43df-b380-b3609f954272 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:18:49.578346 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61468e64-0d51-43df-b380-b3609f954272 None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:49.578630 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61468e64-0d51-43df-b380-b3609f954272 None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:49.578906 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61468e64-0d51-43df-b380-b3609f954272 None admin] PATH_INFO: `/v3/projects` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:49.579714 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61468e64-0d51-43df-b380-b3609f954272 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:18:49.580022 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61468e64-0d51-43df-b380-b3609f954272 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:18:49.581459 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61468e64-0d51-43df-b380-b3609f954272 None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:18:49.593888 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 288/575] 10.222.0.49 () {56 vars in 1148 bytes} [Wed Jun 15 10:18:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:18:49.594659 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61468e64-0d51-43df-b380-b3609f954272 None admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:18:49.605958 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9612c960-bc14-4f3e-a5c6-9b56f05cf32e None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:18:49.638664 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f858fa-f182-47a0-80fb-3d48fd3139e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:18:49.639500 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f858fa-f182-47a0-80fb-3d48fd3139e8 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:49.639791 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f858fa-f182-47a0-80fb-3d48fd3139e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:49.640068 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f858fa-f182-47a0-80fb-3d48fd3139e8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:49.651040 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f858fa-f182-47a0-80fb-3d48fd3139e8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:18:49.653127 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f858fa-f182-47a0-80fb-3d48fd3139e8 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:18:49.670173 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application [None req-f4f858fa-f182-47a0-80fb-3d48fd3139e8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 10:18:49.670173 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:18:49.670173 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:18:49.670173 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:18:49.670173 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:18:49.670173 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:18:49.670173 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:18:49.670173 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:18:49.670173 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:18:49.670173 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:18:49.670173 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:18:49.670173 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:18:49.670173 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 10:18:49.670173 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 10:18:49.670173 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 10:18:49.671568 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 10:18:49.671568 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:18:49.671568 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:18:49.671568 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:18:49.671568 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:18:49.671568 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:18:49.671568 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:18:49.671568 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:18:49.671568 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application with Lock( Jun 15 10:18:49.671568 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:18:49.671568 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:18:49.671568 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:18:49.671568 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:18:49.671568 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:18:49.671568 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:18:49.671568 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:18:49.671568 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:18:49.672482 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 10:18:49.672482 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 10:18:49.672482 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 10:18:49.672482 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Jun 15 10:18:49.672482 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 288/576] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Jun 15 10:18:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:18:49.672482 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f858fa-f182-47a0-80fb-3d48fd3139e8 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:18:49.697274 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61468e64-0d51-43df-b380-b3609f954272 None admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:18:49.731477 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5784bb89-5b29-4f71-bd40-835e5ea13631 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:18:49.732269 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5784bb89-5b29-4f71-bd40-835e5ea13631 None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:49.732553 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5784bb89-5b29-4f71-bd40-835e5ea13631 None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:49.732831 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5784bb89-5b29-4f71-bd40-835e5ea13631 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:49.742831 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5784bb89-5b29-4f71-bd40-835e5ea13631 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 10:18:49.750490 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application [None req-5784bb89-5b29-4f71-bd40-835e5ea13631 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 15 10:18:49.750490 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:18:49.750490 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:18:49.750490 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:18:49.750490 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:18:49.750490 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:18:49.750490 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:18:49.750490 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:18:49.750490 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:18:49.750490 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:18:49.750490 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:18:49.750490 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:18:49.750490 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 10:18:49.750490 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 10:18:49.750490 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 10:18:49.750490 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 10:18:49.751503 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 10:18:49.751503 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 10:18:49.751503 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 10:18:49.751503 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 10:18:49.751503 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:18:49.751503 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:18:49.751503 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:18:49.751503 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:18:49.751503 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:18:49.751503 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:18:49.751503 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:18:49.751503 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application with Lock( Jun 15 10:18:49.751503 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:18:49.751503 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:18:49.751503 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:18:49.751503 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:18:49.752338 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:18:49.752338 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:18:49.752338 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:18:49.752338 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:18:49.752338 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 10:18:49.752338 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 10:18:49.752338 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 10:18:49.752338 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 10:18:49.752338 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 10:18:49.752338 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 10:18:49.752338 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 15 10:18:49.752338 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application Jun 15 10:18:49.752338 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 289/577] 10.222.0.49 () {56 vars in 1102 bytes} [Wed Jun 15 10:18:49 2022] GET /identity/v3/roles/service => generated 85 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:18:49.753024 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5784bb89-5b29-4f71-bd40-835e5ea13631 None admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:18:49.765367 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:18:49.776772 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f858fa-f182-47a0-80fb-3d48fd3139e8 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:18:49.785927 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=100 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:18:49.811351 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea92211a-a864-4a83-b38a-b96182d468ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:18:49.811965 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea92211a-a864-4a83-b38a-b96182d468ac None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:49.812254 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea92211a-a864-4a83-b38a-b96182d468ac None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:49.812533 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea92211a-a864-4a83-b38a-b96182d468ac None admin] PATH_INFO: `/v3/domains` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:49.813259 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea92211a-a864-4a83-b38a-b96182d468ac None admin] RBAC: Adding query filter params (name=Default) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:18:49.813568 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea92211a-a864-4a83-b38a-b96182d468ac None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:18:49.814946 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea92211a-a864-4a83-b38a-b96182d468ac None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:18:49.826853 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 289/578] 10.222.0.49 () {56 vars in 1109 bytes} [Wed Jun 15 10:18:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:18:49.827658 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea92211a-a864-4a83-b38a-b96182d468ac None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:18:49.856912 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5784bb89-5b29-4f71-bd40-835e5ea13631 None admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:18:49.889609 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1523d32-f096-44b4-a0ee-37166ddb9cb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:18:49.890404 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1523d32-f096-44b4-a0ee-37166ddb9cb7 None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:49.890689 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1523d32-f096-44b4-a0ee-37166ddb9cb7 None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:49.890968 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1523d32-f096-44b4-a0ee-37166ddb9cb7 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:49.898309 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1523d32-f096-44b4-a0ee-37166ddb9cb7 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:18:49.899811 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1523d32-f096-44b4-a0ee-37166ddb9cb7 None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:18:49.914273 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application [None req-b1523d32-f096-44b4-a0ee-37166ddb9cb7 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jun 15 10:18:49.914273 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:18:49.914273 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:18:49.914273 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:18:49.914273 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:18:49.914273 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:18:49.914273 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:18:49.914273 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:18:49.914273 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:18:49.914273 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:18:49.914273 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:18:49.914273 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:18:49.914273 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 15 10:18:49.914273 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 15 10:18:49.914273 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 15 10:18:49.914273 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 15 10:18:49.915430 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:18:49.915430 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:18:49.915430 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 15 10:18:49.915430 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 10:18:49.915430 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 15 10:18:49.915430 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 10:18:49.915430 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:18:49.915430 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:18:49.915430 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:18:49.915430 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:18:49.915430 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:18:49.915430 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application with Lock( Jun 15 10:18:49.915430 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:18:49.915430 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:18:49.915430 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:18:49.915430 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:18:49.915430 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:18:49.915430 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:18:49.916420 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:18:49.916420 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:18:49.916420 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 15 10:18:49.916420 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 15 10:18:49.916420 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 15 10:18:49.916420 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 15 10:18:49.916420 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 15 10:18:49.916420 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 15 10:18:49.916420 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jun 15 10:18:49.916420 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application Jun 15 10:18:49.916420 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 290/579] 10.222.0.49 () {56 vars in 1099 bytes} [Wed Jun 15 10:18:49 2022] GET /identity/v3/users/glance => generated 84 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:18:49.916420 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1523d32-f096-44b4-a0ee-37166ddb9cb7 None admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:18:49.927730 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea92211a-a864-4a83-b38a-b96182d468ac None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:18:49.960724 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15a30a5-1e06-4267-8de5-b0b46a527252 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:18:49.961605 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15a30a5-1e06-4267-8de5-b0b46a527252 None admin] REQUEST_METHOD: `PUT` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:49.961890 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15a30a5-1e06-4267-8de5-b0b46a527252 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:49.962165 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15a30a5-1e06-4267-8de5-b0b46a527252 None admin] PATH_INFO: `/v3/projects/559b3e713f5844b6ae37151d56dd419f/users/5ac54528370d4747b6e8777fba9d0c82/roles/16e76452bb5d4ce68a119ad3c9bde41c` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:49.976996 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c15a30a5-1e06-4267-8de5-b0b46a527252 None admin] RBAC: Authorizing `identity:create_grant(project_id=559b3e713f5844b6ae37151d56dd419f, user_id=5ac54528370d4747b6e8777fba9d0c82, role_id=16e76452bb5d4ce68a119ad3c9bde41c)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:18:49.978432 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c15a30a5-1e06-4267-8de5-b0b46a527252 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:18:49.990551 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 290/580] 10.222.0.49 () {58 vars in 1439 bytes} [Wed Jun 15 10:18:49 2022] PUT /identity/v3/projects/559b3e713f5844b6ae37151d56dd419f/users/5ac54528370d4747b6e8777fba9d0c82/roles/16e76452bb5d4ce68a119ad3c9bde41c => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:18:49.991372 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15a30a5-1e06-4267-8de5-b0b46a527252 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:18:50.003152 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:18:50.011637 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=100 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:18:50.024902 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1523d32-f096-44b4-a0ee-37166ddb9cb7 None admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:18:50.066809 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d1f2f9-b20d-4294-aa0e-b686485eb803 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:18:50.067576 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d1f2f9-b20d-4294-aa0e-b686485eb803 None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:50.067853 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d1f2f9-b20d-4294-aa0e-b686485eb803 None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:50.068336 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d1f2f9-b20d-4294-aa0e-b686485eb803 None admin] PATH_INFO: `/v3/domains` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:50.069013 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d1f2f9-b20d-4294-aa0e-b686485eb803 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:18:50.069283 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d1f2f9-b20d-4294-aa0e-b686485eb803 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:18:50.070828 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d1f2f9-b20d-4294-aa0e-b686485eb803 None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:18:50.090710 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 291/581] 10.222.0.49 () {56 vars in 1109 bytes} [Wed Jun 15 10:18:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:18:50.090710 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d1f2f9-b20d-4294-aa0e-b686485eb803 None admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:18:50.116312 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15a30a5-1e06-4267-8de5-b0b46a527252 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:18:50.156111 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3b7ca8-cff8-416c-8f21-341db18c613d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:18:50.156789 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3b7ca8-cff8-416c-8f21-341db18c613d None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:50.157070 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3b7ca8-cff8-416c-8f21-341db18c613d None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:50.157345 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3b7ca8-cff8-416c-8f21-341db18c613d None admin] PATH_INFO: `/v3/roles` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:50.158093 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3b7ca8-cff8-416c-8f21-341db18c613d None admin] RBAC: Adding query filter params (name=service) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:18:50.158394 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3b7ca8-cff8-416c-8f21-341db18c613d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:18:50.160036 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3b7ca8-cff8-416c-8f21-341db18c613d None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:18:50.168561 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 291/582] 10.222.0.49 () {56 vars in 1103 bytes} [Wed Jun 15 10:18:49 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:18:50.169516 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3b7ca8-cff8-416c-8f21-341db18c613d None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:18:50.208178 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d1f2f9-b20d-4294-aa0e-b686485eb803 None admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:18:50.236963 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:18:50.247499 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6964a6c7-5490-44fd-b056-bf7cbb56734a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:18:50.248311 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6964a6c7-5490-44fd-b056-bf7cbb56734a None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:50.248600 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6964a6c7-5490-44fd-b056-bf7cbb56734a None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:50.248881 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6964a6c7-5490-44fd-b056-bf7cbb56734a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:50.257084 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6964a6c7-5490-44fd-b056-bf7cbb56734a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:18:50.258548 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6964a6c7-5490-44fd-b056-bf7cbb56734a None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:18:50.273079 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application [None req-6964a6c7-5490-44fd-b056-bf7cbb56734a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 10:18:50.273079 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:18:50.273079 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:18:50.273079 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:18:50.273079 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:18:50.273079 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:18:50.273079 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:18:50.273079 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:18:50.273079 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:18:50.273079 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:18:50.273079 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:18:50.273079 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:18:50.273079 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 15 10:18:50.273079 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 15 10:18:50.273079 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 15 10:18:50.273969 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 15 10:18:50.273969 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:18:50.273969 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:18:50.273969 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:18:50.273969 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:18:50.273969 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:18:50.273969 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:18:50.273969 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:18:50.273969 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application with Lock( Jun 15 10:18:50.273969 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:18:50.273969 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:18:50.273969 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:18:50.273969 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:18:50.273969 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:18:50.273969 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:18:50.273969 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:18:50.273969 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:18:50.274890 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 15 10:18:50.274890 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 15 10:18:50.274890 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 15 10:18:50.274890 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 15 10:18:50.274890 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 15 10:18:50.274890 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 15 10:18:50.274890 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 10:18:50.274890 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application Jun 15 10:18:50.274890 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 292/583] 10.222.0.49 () {56 vars in 1111 bytes} [Wed Jun 15 10:18:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:18:50.276205 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6964a6c7-5490-44fd-b056-bf7cbb56734a None admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:18:50.281828 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3b7ca8-cff8-416c-8f21-341db18c613d None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:18:50.291716 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=100 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:18:50.319172 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604a5106-4dd9-479a-a888-cb22ab764be7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:18:50.319776 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604a5106-4dd9-479a-a888-cb22ab764be7 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:50.320061 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604a5106-4dd9-479a-a888-cb22ab764be7 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:50.320339 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604a5106-4dd9-479a-a888-cb22ab764be7 None admin] PATH_INFO: `/v3/users` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:50.321171 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-604a5106-4dd9-479a-a888-cb22ab764be7 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:18:50.321480 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-604a5106-4dd9-479a-a888-cb22ab764be7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:18:50.322915 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-604a5106-4dd9-479a-a888-cb22ab764be7 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:18:50.345305 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 292/584] 10.222.0.49 () {56 vars in 1137 bytes} [Wed Jun 15 10:18:50 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:18:50.345305 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604a5106-4dd9-479a-a888-cb22ab764be7 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:18:50.388411 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6964a6c7-5490-44fd-b056-bf7cbb56734a None admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:18:50.425951 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4adbc44e-2d56-4217-83b6-4b4f66403477 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:18:50.426913 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4adbc44e-2d56-4217-83b6-4b4f66403477 None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:50.427232 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4adbc44e-2d56-4217-83b6-4b4f66403477 None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:50.427513 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4adbc44e-2d56-4217-83b6-4b4f66403477 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:50.436944 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4adbc44e-2d56-4217-83b6-4b4f66403477 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:18:50.438930 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4adbc44e-2d56-4217-83b6-4b4f66403477 None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:18:50.446995 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604a5106-4dd9-479a-a888-cb22ab764be7 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:18:50.454840 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application [None req-4adbc44e-2d56-4217-83b6-4b4f66403477 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 15 10:18:50.454840 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:18:50.454840 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:18:50.454840 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:18:50.454840 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:18:50.454840 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:18:50.454840 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:18:50.454840 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:18:50.454840 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:18:50.454840 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:18:50.454840 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:18:50.454840 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:18:50.454840 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 15 10:18:50.454840 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 15 10:18:50.454840 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 15 10:18:50.454840 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 15 10:18:50.455835 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:18:50.455835 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:18:50.455835 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 15 10:18:50.455835 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 10:18:50.455835 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 15 10:18:50.455835 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 10:18:50.455835 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:18:50.455835 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:18:50.455835 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:18:50.455835 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:18:50.455835 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:18:50.455835 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application with Lock( Jun 15 10:18:50.455835 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:18:50.455835 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:18:50.455835 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:18:50.455835 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:18:50.455835 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:18:50.455835 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:18:50.456776 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:18:50.456776 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:18:50.456776 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 15 10:18:50.456776 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 15 10:18:50.456776 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 15 10:18:50.456776 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 15 10:18:50.456776 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 15 10:18:50.456776 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 15 10:18:50.456776 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 15 10:18:50.456776 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application Jun 15 10:18:50.456776 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 293/585] 10.222.0.49 () {56 vars in 1099 bytes} [Wed Jun 15 10:18:50 2022] GET /identity/v3/users/cinder => generated 84 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:18:50.458780 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4adbc44e-2d56-4217-83b6-4b4f66403477 None admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:18:50.477246 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:18:50.491392 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0619b1bc-5130-4d1a-80b0-6a92f09d3757 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:18:50.492255 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0619b1bc-5130-4d1a-80b0-6a92f09d3757 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:50.492558 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0619b1bc-5130-4d1a-80b0-6a92f09d3757 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:50.492831 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0619b1bc-5130-4d1a-80b0-6a92f09d3757 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:50.504084 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0619b1bc-5130-4d1a-80b0-6a92f09d3757 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:18:50.505549 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0619b1bc-5130-4d1a-80b0-6a92f09d3757 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:18:50.524457 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application [None req-0619b1bc-5130-4d1a-80b0-6a92f09d3757 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 10:18:50.524457 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:18:50.524457 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:18:50.524457 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:18:50.524457 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:18:50.524457 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:18:50.524457 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:18:50.524457 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:18:50.524457 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:18:50.524457 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:18:50.524457 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:18:50.524457 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:18:50.524457 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 10:18:50.524457 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 10:18:50.524457 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 10:18:50.525396 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 10:18:50.525396 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:18:50.525396 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:18:50.525396 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:18:50.525396 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:18:50.525396 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:18:50.525396 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:18:50.525396 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:18:50.525396 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application with Lock( Jun 15 10:18:50.525396 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:18:50.525396 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:18:50.525396 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:18:50.525396 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:18:50.525396 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:18:50.525396 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:18:50.525396 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:18:50.525396 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:18:50.526304 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 10:18:50.526304 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 10:18:50.526304 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 10:18:50.526304 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Jun 15 10:18:50.526304 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 293/586] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Jun 15 10:18:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:18:50.526601 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0619b1bc-5130-4d1a-80b0-6a92f09d3757 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:18:50.540048 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=100 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:18:50.574660 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4adbc44e-2d56-4217-83b6-4b4f66403477 None admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:18:50.610542 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a6a3d8-b1a7-40c1-a91e-2e30f6f168f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:18:50.611246 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a6a3d8-b1a7-40c1-a91e-2e30f6f168f7 None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:50.611544 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a6a3d8-b1a7-40c1-a91e-2e30f6f168f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:50.611815 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a6a3d8-b1a7-40c1-a91e-2e30f6f168f7 None admin] PATH_INFO: `/v3/projects` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:50.612601 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a6a3d8-b1a7-40c1-a91e-2e30f6f168f7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:18:50.612913 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a6a3d8-b1a7-40c1-a91e-2e30f6f168f7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:18:50.614349 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a6a3d8-b1a7-40c1-a91e-2e30f6f168f7 None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:18:50.628062 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 294/587] 10.222.0.49 () {56 vars in 1148 bytes} [Wed Jun 15 10:18:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:18:50.628681 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a6a3d8-b1a7-40c1-a91e-2e30f6f168f7 None admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:18:50.634674 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0619b1bc-5130-4d1a-80b0-6a92f09d3757 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:18:50.669914 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ecb980b-fcaf-4b08-a690-de69539fbd9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:18:50.670719 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecb980b-fcaf-4b08-a690-de69539fbd9c None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:50.671311 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecb980b-fcaf-4b08-a690-de69539fbd9c None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:50.671311 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecb980b-fcaf-4b08-a690-de69539fbd9c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:50.682232 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ecb980b-fcaf-4b08-a690-de69539fbd9c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:18:50.684276 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ecb980b-fcaf-4b08-a690-de69539fbd9c None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:18:50.703677 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application [None req-6ecb980b-fcaf-4b08-a690-de69539fbd9c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 10:18:50.703677 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:18:50.703677 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:18:50.703677 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:18:50.703677 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:18:50.703677 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:18:50.703677 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:18:50.703677 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:18:50.703677 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:18:50.703677 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:18:50.703677 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:18:50.703677 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:18:50.703677 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 10:18:50.703677 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 10:18:50.703677 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 10:18:50.704758 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 10:18:50.704758 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:18:50.704758 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:18:50.704758 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:18:50.704758 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:18:50.704758 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:18:50.704758 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:18:50.704758 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:18:50.704758 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application with Lock( Jun 15 10:18:50.704758 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:18:50.704758 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:18:50.704758 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:18:50.704758 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:18:50.704758 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:18:50.704758 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:18:50.704758 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:18:50.704758 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:18:50.705651 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 10:18:50.705651 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 10:18:50.705651 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 10:18:50.705651 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Jun 15 10:18:50.709627 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 294/588] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Jun 15 10:18:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:18:50.710247 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ecb980b-fcaf-4b08-a690-de69539fbd9c None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:18:50.734656 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a6a3d8-b1a7-40c1-a91e-2e30f6f168f7 None admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:18:50.739611 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:18:50.767463 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=100 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:18:50.777635 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a2cb9b-090b-4230-8390-84ed820b7df3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:18:50.778306 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a2cb9b-090b-4230-8390-84ed820b7df3 None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:50.778595 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a2cb9b-090b-4230-8390-84ed820b7df3 None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:50.778871 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a2cb9b-090b-4230-8390-84ed820b7df3 None admin] PATH_INFO: `/v3/users` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:50.779753 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a2cb9b-090b-4230-8390-84ed820b7df3 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:18:50.780079 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a2cb9b-090b-4230-8390-84ed820b7df3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:18:50.781504 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a2cb9b-090b-4230-8390-84ed820b7df3 None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:18:50.806761 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 295/589] 10.222.0.49 () {56 vars in 1137 bytes} [Wed Jun 15 10:18:50 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:18:50.806761 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a2cb9b-090b-4230-8390-84ed820b7df3 None admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:18:50.827376 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ecb980b-fcaf-4b08-a690-de69539fbd9c None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:18:50.864121 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf0ee93-bd7a-41de-b9d3-19c2990d5f87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:18:50.864978 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf0ee93-bd7a-41de-b9d3-19c2990d5f87 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:50.865435 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf0ee93-bd7a-41de-b9d3-19c2990d5f87 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:50.865867 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf0ee93-bd7a-41de-b9d3-19c2990d5f87 None admin] PATH_INFO: `/v3/domains` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:50.866743 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf0ee93-bd7a-41de-b9d3-19c2990d5f87 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:18:50.867112 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf0ee93-bd7a-41de-b9d3-19c2990d5f87 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:18:50.868571 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf0ee93-bd7a-41de-b9d3-19c2990d5f87 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:18:50.884770 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 295/590] 10.222.0.49 () {56 vars in 1109 bytes} [Wed Jun 15 10:18:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:18:50.884770 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf0ee93-bd7a-41de-b9d3-19c2990d5f87 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:18:50.916349 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a2cb9b-090b-4230-8390-84ed820b7df3 None admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:18:50.954570 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26958f2-c1e5-4fdc-8e70-e6871a9731a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:18:50.955258 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26958f2-c1e5-4fdc-8e70-e6871a9731a0 None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:50.955549 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26958f2-c1e5-4fdc-8e70-e6871a9731a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:50.955826 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26958f2-c1e5-4fdc-8e70-e6871a9731a0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:50.956645 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b26958f2-c1e5-4fdc-8e70-e6871a9731a0 None admin] RBAC: Adding query filter params (role.id=16e76452bb5d4ce68a119ad3c9bde41c, scope.project.id=559b3e713f5844b6ae37151d56dd419f, user.id=3a69f99210444f9696e8b22615fb4a41) {{(pid=80119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:18:50.956957 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b26958f2-c1e5-4fdc-8e70-e6871a9731a0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:18:50.958397 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b26958f2-c1e5-4fdc-8e70-e6871a9731a0 None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:18:50.963975 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 296/591] 10.222.0.49 () {56 vars in 1374 bytes} [Wed Jun 15 10:18:50 2022] GET /identity/v3/role_assignments?user.id=3a69f99210444f9696e8b22615fb4a41&scope.project.id=559b3e713f5844b6ae37151d56dd419f&role.id=16e76452bb5d4ce68a119ad3c9bde41c => generated 259 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:18:50.965272 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26958f2-c1e5-4fdc-8e70-e6871a9731a0 None admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:18:50.987756 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:18:51.007878 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf0ee93-bd7a-41de-b9d3-19c2990d5f87 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:18:51.045969 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=100 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:18:51.050446 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3eafe44-4df0-4ab7-a15e-203a07645f90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:18:51.051809 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3eafe44-4df0-4ab7-a15e-203a07645f90 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:51.052330 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3eafe44-4df0-4ab7-a15e-203a07645f90 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:51.053344 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3eafe44-4df0-4ab7-a15e-203a07645f90 None admin] PATH_INFO: `/v3/domains` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:51.054546 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3eafe44-4df0-4ab7-a15e-203a07645f90 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:18:51.055109 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3eafe44-4df0-4ab7-a15e-203a07645f90 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:18:51.056888 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3eafe44-4df0-4ab7-a15e-203a07645f90 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:18:51.071800 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 296/592] 10.222.0.49 () {56 vars in 1109 bytes} [Wed Jun 15 10:18:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:18:51.073242 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3eafe44-4df0-4ab7-a15e-203a07645f90 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:18:51.105503 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26958f2-c1e5-4fdc-8e70-e6871a9731a0 None admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:18:51.150724 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4634d5-abe7-461b-b8c6-6cf868a4a6f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:18:51.152487 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4634d5-abe7-461b-b8c6-6cf868a4a6f5 None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:51.152930 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4634d5-abe7-461b-b8c6-6cf868a4a6f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:51.153355 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4634d5-abe7-461b-b8c6-6cf868a4a6f5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:51.164912 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf4634d5-abe7-461b-b8c6-6cf868a4a6f5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:18:51.166623 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf4634d5-abe7-461b-b8c6-6cf868a4a6f5 None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:18:51.186849 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application [None req-cf4634d5-abe7-461b-b8c6-6cf868a4a6f5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 10:18:51.186849 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:18:51.186849 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:18:51.186849 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:18:51.186849 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:18:51.186849 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:18:51.186849 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:18:51.186849 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:18:51.186849 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:18:51.186849 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:18:51.186849 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:18:51.186849 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:18:51.186849 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 10:18:51.186849 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 10:18:51.186849 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 10:18:51.188032 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 10:18:51.188032 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:18:51.188032 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:18:51.188032 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:18:51.188032 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:18:51.188032 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:18:51.188032 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:18:51.188032 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:18:51.188032 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application with Lock( Jun 15 10:18:51.188032 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:18:51.188032 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:18:51.188032 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:18:51.188032 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:18:51.188032 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:18:51.188032 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:18:51.188032 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:18:51.188032 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:18:51.189001 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 10:18:51.189001 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 10:18:51.189001 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 10:18:51.189001 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application Jun 15 10:18:51.189001 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 297/593] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Jun 15 10:18:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:18:51.190655 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4634d5-abe7-461b-b8c6-6cf868a4a6f5 None admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:18:51.206131 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3eafe44-4df0-4ab7-a15e-203a07645f90 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:18:51.247021 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10cfa4a-b10f-4629-b006-dbe7324eb52a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:18:51.248183 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10cfa4a-b10f-4629-b006-dbe7324eb52a None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:51.248619 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10cfa4a-b10f-4629-b006-dbe7324eb52a None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:51.249046 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10cfa4a-b10f-4629-b006-dbe7324eb52a None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:51.257240 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d10cfa4a-b10f-4629-b006-dbe7324eb52a None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:18:51.258910 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d10cfa4a-b10f-4629-b006-dbe7324eb52a None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:18:51.276718 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application [None req-d10cfa4a-b10f-4629-b006-dbe7324eb52a None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jun 15 10:18:51.276718 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:18:51.276718 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:18:51.276718 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:18:51.276718 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:18:51.276718 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:18:51.276718 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:18:51.276718 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:18:51.276718 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:18:51.276718 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:18:51.276718 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:18:51.276718 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:18:51.276718 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 15 10:18:51.276718 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 15 10:18:51.276718 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 15 10:18:51.276718 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 15 10:18:51.278408 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:18:51.278408 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:18:51.278408 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 15 10:18:51.278408 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 10:18:51.278408 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 15 10:18:51.278408 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 10:18:51.278408 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:18:51.278408 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:18:51.278408 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:18:51.278408 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:18:51.278408 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:18:51.278408 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application with Lock( Jun 15 10:18:51.278408 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:18:51.278408 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:18:51.278408 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:18:51.278408 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:18:51.278408 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:18:51.278408 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:18:51.279553 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:18:51.279553 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:18:51.279553 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 15 10:18:51.279553 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 15 10:18:51.279553 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 15 10:18:51.279553 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 15 10:18:51.279553 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 15 10:18:51.279553 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 15 10:18:51.279553 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jun 15 10:18:51.279553 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Jun 15 10:18:51.281827 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 297/594] 10.222.0.49 () {56 vars in 1102 bytes} [Wed Jun 15 10:18:51 2022] GET /identity/v3/users/neutron => generated 85 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:18:51.283062 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10cfa4a-b10f-4629-b006-dbe7324eb52a None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:18:51.291723 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4634d5-abe7-461b-b8c6-6cf868a4a6f5 None admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:18:51.292617 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:18:51.299161 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=100 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:18:51.327663 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9add18-8890-44a6-a42a-28cd25f5359f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:18:51.328465 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9add18-8890-44a6-a42a-28cd25f5359f None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:51.328749 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9add18-8890-44a6-a42a-28cd25f5359f None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:51.329024 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9add18-8890-44a6-a42a-28cd25f5359f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:51.336305 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9add18-8890-44a6-a42a-28cd25f5359f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:18:51.337763 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9add18-8890-44a6-a42a-28cd25f5359f None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:18:51.353456 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application [None req-ec9add18-8890-44a6-a42a-28cd25f5359f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 10:18:51.353456 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:18:51.353456 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:18:51.353456 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:18:51.353456 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:18:51.353456 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:18:51.353456 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:18:51.353456 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:18:51.353456 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:18:51.353456 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:18:51.353456 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:18:51.353456 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:18:51.353456 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 15 10:18:51.353456 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 15 10:18:51.353456 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 15 10:18:51.354579 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 15 10:18:51.354579 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:18:51.354579 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:18:51.354579 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:18:51.354579 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:18:51.354579 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:18:51.354579 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:18:51.354579 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:18:51.354579 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application with Lock( Jun 15 10:18:51.354579 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:18:51.354579 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:18:51.354579 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:18:51.354579 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:18:51.354579 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:18:51.354579 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:18:51.354579 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:18:51.354579 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:18:51.355731 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 15 10:18:51.355731 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 15 10:18:51.355731 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 15 10:18:51.355731 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 15 10:18:51.355731 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 15 10:18:51.355731 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 15 10:18:51.355731 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 10:18:51.355731 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application Jun 15 10:18:51.359345 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 298/595] 10.222.0.49 () {56 vars in 1111 bytes} [Wed Jun 15 10:18:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:18:51.360304 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9add18-8890-44a6-a42a-28cd25f5359f None admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:18:51.398965 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10cfa4a-b10f-4629-b006-dbe7324eb52a None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:18:51.435821 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738219f5-4327-4d6d-90e6-119f7268609d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:18:51.436763 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738219f5-4327-4d6d-90e6-119f7268609d None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:51.437227 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738219f5-4327-4d6d-90e6-119f7268609d None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:51.437655 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738219f5-4327-4d6d-90e6-119f7268609d None admin] PATH_INFO: `/v3/domains` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:51.438723 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-738219f5-4327-4d6d-90e6-119f7268609d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:18:51.439208 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-738219f5-4327-4d6d-90e6-119f7268609d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:18:51.440831 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-738219f5-4327-4d6d-90e6-119f7268609d None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:18:51.454019 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 298/596] 10.222.0.49 () {56 vars in 1109 bytes} [Wed Jun 15 10:18:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:18:51.455293 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738219f5-4327-4d6d-90e6-119f7268609d None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:18:51.468612 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9add18-8890-44a6-a42a-28cd25f5359f None admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:18:51.508977 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b392858e-5687-4056-87c6-3ee37b22e768 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:18:51.510398 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b392858e-5687-4056-87c6-3ee37b22e768 None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:51.510857 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b392858e-5687-4056-87c6-3ee37b22e768 None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:51.512490 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b392858e-5687-4056-87c6-3ee37b22e768 None admin] PATH_INFO: `/v3/users` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:51.513898 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b392858e-5687-4056-87c6-3ee37b22e768 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:18:51.514681 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b392858e-5687-4056-87c6-3ee37b22e768 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:18:51.516613 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b392858e-5687-4056-87c6-3ee37b22e768 None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:18:51.535565 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:18:51.542936 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 299/597] 10.222.0.49 () {56 vars in 1139 bytes} [Wed Jun 15 10:18:51 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:18:51.544280 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b392858e-5687-4056-87c6-3ee37b22e768 None admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:18:51.576725 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738219f5-4327-4d6d-90e6-119f7268609d None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:18:51.581213 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=100 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:18:51.623245 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5860deae-2215-48d5-815e-77ed5c05e573 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:18:51.624213 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5860deae-2215-48d5-815e-77ed5c05e573 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:51.624578 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5860deae-2215-48d5-815e-77ed5c05e573 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:51.624926 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5860deae-2215-48d5-815e-77ed5c05e573 None admin] PATH_INFO: `/v3/projects` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:51.626054 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5860deae-2215-48d5-815e-77ed5c05e573 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:18:51.626434 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5860deae-2215-48d5-815e-77ed5c05e573 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:18:51.628686 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5860deae-2215-48d5-815e-77ed5c05e573 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:18:51.646421 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 299/598] 10.222.0.49 () {56 vars in 1148 bytes} [Wed Jun 15 10:18:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:18:51.647446 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5860deae-2215-48d5-815e-77ed5c05e573 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:18:51.686272 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b392858e-5687-4056-87c6-3ee37b22e768 None admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:18:51.733139 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4217db86-57e7-4822-a612-3e801d8daa39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:18:51.734856 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4217db86-57e7-4822-a612-3e801d8daa39 None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:51.735360 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4217db86-57e7-4822-a612-3e801d8daa39 None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:51.735832 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4217db86-57e7-4822-a612-3e801d8daa39 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:51.746044 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4217db86-57e7-4822-a612-3e801d8daa39 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:18:51.747865 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4217db86-57e7-4822-a612-3e801d8daa39 None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:18:51.760914 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5860deae-2215-48d5-815e-77ed5c05e573 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:18:51.767931 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application [None req-4217db86-57e7-4822-a612-3e801d8daa39 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 10:18:51.767931 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:18:51.767931 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:18:51.767931 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:18:51.767931 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:18:51.767931 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:18:51.767931 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:18:51.767931 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:18:51.767931 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:18:51.767931 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:18:51.767931 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:18:51.767931 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:18:51.767931 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 15 10:18:51.767931 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 15 10:18:51.767931 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 15 10:18:51.768822 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 15 10:18:51.768822 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:18:51.768822 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:18:51.768822 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:18:51.768822 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:18:51.768822 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:18:51.768822 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:18:51.768822 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:18:51.768822 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application with Lock( Jun 15 10:18:51.768822 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:18:51.768822 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:18:51.768822 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:18:51.768822 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:18:51.768822 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:18:51.768822 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:18:51.768822 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:18:51.768822 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:18:51.769765 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 15 10:18:51.769765 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 15 10:18:51.769765 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 15 10:18:51.769765 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 15 10:18:51.769765 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 15 10:18:51.769765 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 15 10:18:51.769765 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 10:18:51.769765 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application Jun 15 10:18:51.775468 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 300/599] 10.222.0.49 () {56 vars in 1111 bytes} [Wed Jun 15 10:18:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:18:51.776479 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4217db86-57e7-4822-a612-3e801d8daa39 None admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:18:51.805305 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:18:51.817148 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ee86e1-9d6b-4e7f-a484-c4fedeab92c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:18:51.818353 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ee86e1-9d6b-4e7f-a484-c4fedeab92c4 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:51.818844 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ee86e1-9d6b-4e7f-a484-c4fedeab92c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:51.819373 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ee86e1-9d6b-4e7f-a484-c4fedeab92c4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:51.831130 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ee86e1-9d6b-4e7f-a484-c4fedeab92c4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:18:51.832909 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ee86e1-9d6b-4e7f-a484-c4fedeab92c4 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:18:51.851248 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application [None req-75ee86e1-9d6b-4e7f-a484-c4fedeab92c4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 10:18:51.851248 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:18:51.851248 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:18:51.851248 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:18:51.851248 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:18:51.851248 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:18:51.851248 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:18:51.851248 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:18:51.851248 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:18:51.851248 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:18:51.851248 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:18:51.851248 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:18:51.851248 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 10:18:51.851248 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 10:18:51.851248 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 10:18:51.852144 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 10:18:51.852144 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:18:51.852144 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:18:51.852144 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:18:51.852144 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:18:51.852144 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:18:51.852144 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:18:51.852144 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:18:51.852144 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application with Lock( Jun 15 10:18:51.852144 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:18:51.852144 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:18:51.852144 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:18:51.852144 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:18:51.852144 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:18:51.852144 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:18:51.852144 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:18:51.852144 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:18:51.853058 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 10:18:51.853058 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 10:18:51.853058 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 10:18:51.853058 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Jun 15 10:18:51.855430 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 300/600] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Jun 15 10:18:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:18:51.856483 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ee86e1-9d6b-4e7f-a484-c4fedeab92c4 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:18:51.877749 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=100 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:18:51.895698 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4217db86-57e7-4822-a612-3e801d8daa39 None admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:18:51.930421 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9671cc96-55b2-4e1e-b305-25526f32c3c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:18:51.931328 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9671cc96-55b2-4e1e-b305-25526f32c3c4 None admin] REQUEST_METHOD: `PUT` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:51.931615 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9671cc96-55b2-4e1e-b305-25526f32c3c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:51.931894 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9671cc96-55b2-4e1e-b305-25526f32c3c4 None admin] PATH_INFO: `/v3/projects/559b3e713f5844b6ae37151d56dd419f/users/91f2fe3be62e49548eae208e57bf2a93/roles/16e76452bb5d4ce68a119ad3c9bde41c` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:51.947980 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9671cc96-55b2-4e1e-b305-25526f32c3c4 None admin] RBAC: Authorizing `identity:create_grant(project_id=559b3e713f5844b6ae37151d56dd419f, user_id=91f2fe3be62e49548eae208e57bf2a93, role_id=16e76452bb5d4ce68a119ad3c9bde41c)` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:18:51.949474 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9671cc96-55b2-4e1e-b305-25526f32c3c4 None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:18:51.966013 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 301/601] 10.222.0.49 () {58 vars in 1439 bytes} [Wed Jun 15 10:18:51 2022] PUT /identity/v3/projects/559b3e713f5844b6ae37151d56dd419f/users/91f2fe3be62e49548eae208e57bf2a93/roles/16e76452bb5d4ce68a119ad3c9bde41c => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:18:51.966921 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9671cc96-55b2-4e1e-b305-25526f32c3c4 None admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:18:51.967932 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ee86e1-9d6b-4e7f-a484-c4fedeab92c4 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:18:52.010839 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3ba831-263e-4067-98c9-3d4fce02d259 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:18:52.012057 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3ba831-263e-4067-98c9-3d4fce02d259 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:52.012552 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3ba831-263e-4067-98c9-3d4fce02d259 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:52.013086 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3ba831-263e-4067-98c9-3d4fce02d259 None admin] PATH_INFO: `/v3/projects` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:52.014208 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff3ba831-263e-4067-98c9-3d4fce02d259 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:18:52.014800 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff3ba831-263e-4067-98c9-3d4fce02d259 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:18:52.016643 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff3ba831-263e-4067-98c9-3d4fce02d259 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:18:52.035361 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 301/602] 10.222.0.49 () {56 vars in 1148 bytes} [Wed Jun 15 10:18:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:18:52.037635 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3ba831-263e-4067-98c9-3d4fce02d259 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:18:52.079499 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9671cc96-55b2-4e1e-b305-25526f32c3c4 None admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:18:52.084744 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:18:52.120337 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b840f840-f27f-4668-87a3-53313f04a080 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:18:52.121240 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b840f840-f27f-4668-87a3-53313f04a080 None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:52.121465 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b840f840-f27f-4668-87a3-53313f04a080 None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:52.121745 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b840f840-f27f-4668-87a3-53313f04a080 None admin] PATH_INFO: `/v3/domains` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:52.122508 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b840f840-f27f-4668-87a3-53313f04a080 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:18:52.122815 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b840f840-f27f-4668-87a3-53313f04a080 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:18:52.124298 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b840f840-f27f-4668-87a3-53313f04a080 None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:18:52.126088 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:18:52.138603 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 302/603] 10.222.0.49 () {56 vars in 1109 bytes} [Wed Jun 15 10:18:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:18:52.145324 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b840f840-f27f-4668-87a3-53313f04a080 None admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:18:52.167886 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3ba831-263e-4067-98c9-3d4fce02d259 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:18:52.209971 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062e713c-0a7b-4d4b-9a5e-9206898302a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:18:52.211436 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062e713c-0a7b-4d4b-9a5e-9206898302a1 None admin] REQUEST_METHOD: `PUT` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:52.212027 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062e713c-0a7b-4d4b-9a5e-9206898302a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:52.212539 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062e713c-0a7b-4d4b-9a5e-9206898302a1 None admin] PATH_INFO: `/v3/projects/559b3e713f5844b6ae37151d56dd419f/users/20d1327553404b80aecee87c78fcb52b/roles/16e76452bb5d4ce68a119ad3c9bde41c` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:52.243311 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-062e713c-0a7b-4d4b-9a5e-9206898302a1 None admin] RBAC: Authorizing `identity:create_grant(project_id=559b3e713f5844b6ae37151d56dd419f, user_id=20d1327553404b80aecee87c78fcb52b, role_id=16e76452bb5d4ce68a119ad3c9bde41c)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:18:52.245233 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-062e713c-0a7b-4d4b-9a5e-9206898302a1 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:18:52.264311 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 302/604] 10.222.0.49 () {58 vars in 1439 bytes} [Wed Jun 15 10:18:52 2022] PUT /identity/v3/projects/559b3e713f5844b6ae37151d56dd419f/users/20d1327553404b80aecee87c78fcb52b/roles/16e76452bb5d4ce68a119ad3c9bde41c => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:18:52.266203 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da9511a-8ca5-4c10-8d8f-e32606067ac9 None None] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:52.266772 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da9511a-8ca5-4c10-8d8f-e32606067ac9 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:52.271649 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da9511a-8ca5-4c10-8d8f-e32606067ac9 None None] PATH_INFO: `/` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:52.273713 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 303/605] 10.222.0.49 () {54 vars in 926 bytes} [Wed Jun 15 10:18:52 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:18:52.279733 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2c7bb8-7cfb-4511-9e8c-4f3840a96002 None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:52.280358 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2c7bb8-7cfb-4511-9e8c-4f3840a96002 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:52.280865 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2c7bb8-7cfb-4511-9e8c-4f3840a96002 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:52.287700 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b840f840-f27f-4668-87a3-53313f04a080 None admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:18:52.308356 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-1d2c7bb8-7cfb-4511-9e8c-4f3840a96002 None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:18:52.310115 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-1d2c7bb8-7cfb-4511-9e8c-4f3840a96002 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:18:52.327148 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb7a368-7db8-4b0f-a39a-79f0aafa20b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:18:52.327148 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb7a368-7db8-4b0f-a39a-79f0aafa20b4 None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:52.327148 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb7a368-7db8-4b0f-a39a-79f0aafa20b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:52.327148 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb7a368-7db8-4b0f-a39a-79f0aafa20b4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:52.333616 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb7a368-7db8-4b0f-a39a-79f0aafa20b4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:18:52.335137 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb7a368-7db8-4b0f-a39a-79f0aafa20b4 None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:18:52.351662 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application [None req-4fb7a368-7db8-4b0f-a39a-79f0aafa20b4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 10:18:52.351662 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:18:52.351662 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:18:52.351662 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:18:52.351662 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:18:52.351662 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:18:52.351662 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:18:52.351662 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:18:52.351662 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:18:52.351662 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:18:52.351662 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:18:52.351662 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:18:52.351662 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 15 10:18:52.351662 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 15 10:18:52.351662 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 15 10:18:52.352719 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 15 10:18:52.352719 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:18:52.352719 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:18:52.352719 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:18:52.352719 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:18:52.352719 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:18:52.352719 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:18:52.352719 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:18:52.352719 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application with Lock( Jun 15 10:18:52.352719 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:18:52.352719 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:18:52.352719 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:18:52.352719 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:18:52.352719 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:18:52.352719 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:18:52.352719 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:18:52.352719 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:18:52.353647 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 15 10:18:52.353647 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 15 10:18:52.353647 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 15 10:18:52.353647 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 15 10:18:52.353647 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 15 10:18:52.353647 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 15 10:18:52.353647 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 10:18:52.353647 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application Jun 15 10:18:52.353647 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 303/606] 10.222.0.49 () {56 vars in 1111 bytes} [Wed Jun 15 10:18:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:18:52.361343 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb7a368-7db8-4b0f-a39a-79f0aafa20b4 None admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:18:52.422898 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 304/607] 10.222.0.49 () {58 vars in 1023 bytes} [Wed Jun 15 10:18:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 10:18:52.463712 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=100 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:18:52.466762 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2c7bb8-7cfb-4511-9e8c-4f3840a96002 None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:18:52.492507 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:18:52.520856 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb7a368-7db8-4b0f-a39a-79f0aafa20b4 None admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:18:52.559404 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e881ef8c-0819-48a6-8c7f-e7ae43b1b9c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:18:52.560212 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e881ef8c-0819-48a6-8c7f-e7ae43b1b9c5 None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:52.560593 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e881ef8c-0819-48a6-8c7f-e7ae43b1b9c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:52.560965 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e881ef8c-0819-48a6-8c7f-e7ae43b1b9c5 None admin] PATH_INFO: `/v3/projects` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:52.561849 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e881ef8c-0819-48a6-8c7f-e7ae43b1b9c5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:18:52.562254 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e881ef8c-0819-48a6-8c7f-e7ae43b1b9c5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:18:52.564531 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e881ef8c-0819-48a6-8c7f-e7ae43b1b9c5 None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:18:52.576923 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 304/608] 10.222.0.49 () {56 vars in 1148 bytes} [Wed Jun 15 10:18:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:18:52.582542 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e881ef8c-0819-48a6-8c7f-e7ae43b1b9c5 None admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:18:52.585054 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2c7bb8-7cfb-4511-9e8c-4f3840a96002 None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:18:52.641087 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41de61e-cf59-4b99-b6ea-c9d4d1f8f398 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:18:52.641896 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41de61e-cf59-4b99-b6ea-c9d4d1f8f398 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:52.642175 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41de61e-cf59-4b99-b6ea-c9d4d1f8f398 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:52.642448 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41de61e-cf59-4b99-b6ea-c9d4d1f8f398 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:52.667862 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41de61e-cf59-4b99-b6ea-c9d4d1f8f398 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 10:18:52.675896 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application [None req-c41de61e-cf59-4b99-b6ea-c9d4d1f8f398 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 15 10:18:52.675896 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:18:52.675896 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:18:52.675896 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:18:52.675896 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:18:52.675896 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:18:52.675896 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:18:52.675896 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:18:52.675896 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:18:52.675896 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:18:52.675896 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:18:52.675896 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:18:52.675896 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 10:18:52.675896 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 10:18:52.675896 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 10:18:52.675896 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 10:18:52.677087 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 10:18:52.677087 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 10:18:52.677087 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 10:18:52.677087 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 10:18:52.677087 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:18:52.677087 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:18:52.677087 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:18:52.677087 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:18:52.677087 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:18:52.677087 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:18:52.677087 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:18:52.677087 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application with Lock( Jun 15 10:18:52.677087 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:18:52.677087 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:18:52.677087 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:18:52.677087 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:18:52.678125 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:18:52.678125 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:18:52.678125 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:18:52.678125 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:18:52.678125 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 10:18:52.678125 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 10:18:52.678125 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 10:18:52.678125 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 10:18:52.678125 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 10:18:52.678125 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 10:18:52.678125 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 15 10:18:52.678125 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Jun 15 10:18:52.687228 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 305/609] 10.222.0.49 () {56 vars in 1102 bytes} [Wed Jun 15 10:18:52 2022] GET /identity/v3/roles/service => generated 85 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 10:18:52.693477 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41de61e-cf59-4b99-b6ea-c9d4d1f8f398 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:18:52.733218 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e881ef8c-0819-48a6-8c7f-e7ae43b1b9c5 None admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:18:52.749188 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=100 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:18:52.782739 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c5a88d-8390-4227-a552-322b4e1e7435 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:18:52.783679 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c5a88d-8390-4227-a552-322b4e1e7435 None admin] REQUEST_METHOD: `PUT` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:52.783967 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c5a88d-8390-4227-a552-322b4e1e7435 None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:52.784246 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c5a88d-8390-4227-a552-322b4e1e7435 None admin] PATH_INFO: `/v3/projects/559b3e713f5844b6ae37151d56dd419f/users/0cb44a17d205431793e9a8791e477a32/roles/16e76452bb5d4ce68a119ad3c9bde41c` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:52.822050 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c5a88d-8390-4227-a552-322b4e1e7435 None admin] RBAC: Authorizing `identity:create_grant(project_id=559b3e713f5844b6ae37151d56dd419f, user_id=0cb44a17d205431793e9a8791e477a32, role_id=16e76452bb5d4ce68a119ad3c9bde41c)` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:18:52.824510 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c5a88d-8390-4227-a552-322b4e1e7435 None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:18:52.850703 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:18:52.852595 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 305/610] 10.222.0.49 () {58 vars in 1439 bytes} [Wed Jun 15 10:18:52 2022] PUT /identity/v3/projects/559b3e713f5844b6ae37151d56dd419f/users/0cb44a17d205431793e9a8791e477a32/roles/16e76452bb5d4ce68a119ad3c9bde41c => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 10:18:52.878844 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41de61e-cf59-4b99-b6ea-c9d4d1f8f398 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:18:52.920317 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac91db9-9073-4ac2-8a45-a9e3008f35e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:18:52.921340 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac91db9-9073-4ac2-8a45-a9e3008f35e1 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:52.921780 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac91db9-9073-4ac2-8a45-a9e3008f35e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:52.922260 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac91db9-9073-4ac2-8a45-a9e3008f35e1 None admin] PATH_INFO: `/v3/roles` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:52.923204 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac91db9-9073-4ac2-8a45-a9e3008f35e1 None admin] RBAC: Adding query filter params (name=service) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:18:52.923668 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac91db9-9073-4ac2-8a45-a9e3008f35e1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:18:52.925305 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac91db9-9073-4ac2-8a45-a9e3008f35e1 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:18:52.935869 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 306/611] 10.222.0.49 () {56 vars in 1103 bytes} [Wed Jun 15 10:18:52 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:18:52.944268 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c5a88d-8390-4227-a552-322b4e1e7435 None admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:18:53.061891 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c5a88d-8390-4227-a552-322b4e1e7435 None admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:18:53.103909 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248105c0-f38d-4503-92ff-2328d1fba3e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:18:53.105236 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248105c0-f38d-4503-92ff-2328d1fba3e0 None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:53.105813 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248105c0-f38d-4503-92ff-2328d1fba3e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:53.106390 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248105c0-f38d-4503-92ff-2328d1fba3e0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:53.118363 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248105c0-f38d-4503-92ff-2328d1fba3e0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:18:53.120079 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248105c0-f38d-4503-92ff-2328d1fba3e0 None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:18:53.139768 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application [None req-248105c0-f38d-4503-92ff-2328d1fba3e0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 10:18:53.139768 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:18:53.139768 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:18:53.139768 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:18:53.139768 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:18:53.139768 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:18:53.139768 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:18:53.139768 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:18:53.139768 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:18:53.139768 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:18:53.139768 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:18:53.139768 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:18:53.139768 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 10:18:53.139768 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 10:18:53.139768 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 10:18:53.140954 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 10:18:53.140954 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:18:53.140954 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:18:53.140954 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:18:53.140954 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:18:53.140954 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:18:53.140954 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:18:53.140954 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:18:53.140954 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application with Lock( Jun 15 10:18:53.140954 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:18:53.140954 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:18:53.140954 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:18:53.140954 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:18:53.140954 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:18:53.140954 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:18:53.140954 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:18:53.140954 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:18:53.142057 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 10:18:53.142057 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 10:18:53.142057 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 10:18:53.142057 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application Jun 15 10:18:53.143956 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 306/612] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Jun 15 10:18:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:18:53.149143 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac91db9-9073-4ac2-8a45-a9e3008f35e1 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:18:53.264687 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac91db9-9073-4ac2-8a45-a9e3008f35e1 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:18:53.301112 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460bf2c4-af2f-4ae0-bd10-7c702ce55c32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:18:53.302097 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460bf2c4-af2f-4ae0-bd10-7c702ce55c32 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:53.302578 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460bf2c4-af2f-4ae0-bd10-7c702ce55c32 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:53.303033 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460bf2c4-af2f-4ae0-bd10-7c702ce55c32 None admin] PATH_INFO: `/v3/domains` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:53.303961 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-460bf2c4-af2f-4ae0-bd10-7c702ce55c32 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:18:53.304428 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-460bf2c4-af2f-4ae0-bd10-7c702ce55c32 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:18:53.305942 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-460bf2c4-af2f-4ae0-bd10-7c702ce55c32 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:18:53.315812 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:18:53.320126 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 307/613] 10.222.0.49 () {56 vars in 1109 bytes} [Wed Jun 15 10:18:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:18:53.328165 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248105c0-f38d-4503-92ff-2328d1fba3e0 None admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:18:53.363219 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:18:53.479736 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248105c0-f38d-4503-92ff-2328d1fba3e0 None admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:18:53.529113 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01aad5e7-4c48-42d6-85ab-1995f375ad5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:18:53.529900 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01aad5e7-4c48-42d6-85ab-1995f375ad5c None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:53.530183 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01aad5e7-4c48-42d6-85ab-1995f375ad5c None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:53.530508 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01aad5e7-4c48-42d6-85ab-1995f375ad5c None admin] PATH_INFO: `/v3/users/nova` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:53.540985 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01aad5e7-4c48-42d6-85ab-1995f375ad5c None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:18:53.542492 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01aad5e7-4c48-42d6-85ab-1995f375ad5c None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:18:53.548296 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d3f344-c724-432b-b610-cb3552777492 None None] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:53.548589 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d3f344-c724-432b-b610-cb3552777492 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:53.548865 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d3f344-c724-432b-b610-cb3552777492 None None] PATH_INFO: `/` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:53.550550 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 308/614] 10.222.0.49 () {54 vars in 926 bytes} [Wed Jun 15 10:18:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:18:53.559891 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b7355e-83da-4711-93e1-89674e105c85 None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:53.560175 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b7355e-83da-4711-93e1-89674e105c85 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:53.560448 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b7355e-83da-4711-93e1-89674e105c85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:53.567063 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application [None req-01aad5e7-4c48-42d6-85ab-1995f375ad5c None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 15 10:18:53.567063 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:18:53.567063 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:18:53.567063 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:18:53.567063 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:18:53.567063 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:18:53.567063 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:18:53.567063 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:18:53.567063 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:18:53.567063 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:18:53.567063 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:18:53.567063 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:18:53.567063 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 15 10:18:53.567063 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 15 10:18:53.567063 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 15 10:18:53.567063 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 15 10:18:53.568051 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:18:53.568051 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:18:53.568051 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 15 10:18:53.568051 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 10:18:53.568051 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 15 10:18:53.568051 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 10:18:53.568051 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:18:53.568051 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:18:53.568051 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:18:53.568051 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:18:53.568051 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:18:53.568051 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application with Lock( Jun 15 10:18:53.568051 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:18:53.568051 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:18:53.568051 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:18:53.568051 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:18:53.568051 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:18:53.568051 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:18:53.570550 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:18:53.570550 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:18:53.570550 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 15 10:18:53.570550 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 15 10:18:53.570550 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 15 10:18:53.570550 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 15 10:18:53.570550 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 15 10:18:53.570550 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 15 10:18:53.570550 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 15 10:18:53.570550 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application Jun 15 10:18:53.570550 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 307/615] 10.222.0.49 () {56 vars in 1093 bytes} [Wed Jun 15 10:18:53 2022] GET /identity/v3/users/nova => generated 82 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:18:53.574476 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01aad5e7-4c48-42d6-85ab-1995f375ad5c None admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:18:53.592054 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-b8b7355e-83da-4711-93e1-89674e105c85 None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:18:53.593535 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-b8b7355e-83da-4711-93e1-89674e105c85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:18:53.681034 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=100 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:18:53.708312 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01aad5e7-4c48-42d6-85ab-1995f375ad5c None admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:18:53.756479 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 309/616] 10.222.0.49 () {58 vars in 1023 bytes} [Wed Jun 15 10:18:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 10:18:53.758022 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846e4caa-e46e-41a1-90f2-9b8af6abac79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:18:53.758760 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846e4caa-e46e-41a1-90f2-9b8af6abac79 None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:53.759051 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846e4caa-e46e-41a1-90f2-9b8af6abac79 None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:53.759400 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846e4caa-e46e-41a1-90f2-9b8af6abac79 None admin] PATH_INFO: `/v3/users` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:53.760247 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-846e4caa-e46e-41a1-90f2-9b8af6abac79 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:18:53.760556 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-846e4caa-e46e-41a1-90f2-9b8af6abac79 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:18:53.762554 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-846e4caa-e46e-41a1-90f2-9b8af6abac79 None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:18:53.779816 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b7355e-83da-4711-93e1-89674e105c85 None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:18:53.807170 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 308/617] 10.222.0.49 () {56 vars in 1133 bytes} [Wed Jun 15 10:18:53 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 10:18:53.816011 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846e4caa-e46e-41a1-90f2-9b8af6abac79 None admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:18:53.951898 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b7355e-83da-4711-93e1-89674e105c85 None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:18:53.988442 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846e4caa-e46e-41a1-90f2-9b8af6abac79 None admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:18:54.007957 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:18:54.016897 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97082d0d-5f76-42e9-ab7d-ec9838dfe6a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:18:54.017802 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97082d0d-5f76-42e9-ab7d-ec9838dfe6a8 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:54.018087 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97082d0d-5f76-42e9-ab7d-ec9838dfe6a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:54.018362 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97082d0d-5f76-42e9-ab7d-ec9838dfe6a8 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:54.029293 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97082d0d-5f76-42e9-ab7d-ec9838dfe6a8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 10:18:54.037523 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application [None req-97082d0d-5f76-42e9-ab7d-ec9838dfe6a8 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 15 10:18:54.037523 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:18:54.037523 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:18:54.037523 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:18:54.037523 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:18:54.037523 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:18:54.037523 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:18:54.037523 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:18:54.037523 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:18:54.037523 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:18:54.037523 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:18:54.037523 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:18:54.037523 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 10:18:54.037523 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 10:18:54.037523 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 10:18:54.037523 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 10:18:54.038447 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 10:18:54.038447 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 10:18:54.038447 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 10:18:54.038447 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 10:18:54.038447 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:18:54.038447 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:18:54.038447 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:18:54.038447 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:18:54.038447 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:18:54.038447 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:18:54.038447 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:18:54.038447 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application with Lock( Jun 15 10:18:54.038447 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:18:54.038447 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:18:54.038447 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:18:54.038447 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:18:54.039342 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:18:54.039342 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:18:54.039342 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:18:54.039342 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:18:54.039342 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 10:18:54.039342 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 10:18:54.039342 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 10:18:54.039342 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 10:18:54.039342 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 10:18:54.039342 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 10:18:54.039342 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 15 10:18:54.039342 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Jun 15 10:18:54.043242 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 310/618] 10.222.0.49 () {56 vars in 1102 bytes} [Wed Jun 15 10:18:53 2022] GET /identity/v3/roles/service => generated 85 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 10:18:54.045032 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97082d0d-5f76-42e9-ab7d-ec9838dfe6a8 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:18:54.060723 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b496b1-0ddc-4523-a2f3-039ec7d1f83a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:18:54.061400 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b496b1-0ddc-4523-a2f3-039ec7d1f83a None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:54.061688 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b496b1-0ddc-4523-a2f3-039ec7d1f83a None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:54.061968 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b496b1-0ddc-4523-a2f3-039ec7d1f83a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:54.069454 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=100 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:18:54.162966 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b496b1-0ddc-4523-a2f3-039ec7d1f83a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:18:54.164477 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b496b1-0ddc-4523-a2f3-039ec7d1f83a None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:18:54.194744 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97082d0d-5f76-42e9-ab7d-ec9838dfe6a8 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:18:54.261776 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c49f09-1b8f-4d4d-a1cd-b193a996707f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:18:54.262861 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c49f09-1b8f-4d4d-a1cd-b193a996707f None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:54.263473 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c49f09-1b8f-4d4d-a1cd-b193a996707f None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:54.263955 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c49f09-1b8f-4d4d-a1cd-b193a996707f None admin] PATH_INFO: `/v3/roles` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:54.264943 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c49f09-1b8f-4d4d-a1cd-b193a996707f None admin] RBAC: Adding query filter params (name=service) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:18:54.265458 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c49f09-1b8f-4d4d-a1cd-b193a996707f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:18:54.267049 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c49f09-1b8f-4d4d-a1cd-b193a996707f None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:18:54.282243 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 311/619] 10.222.0.49 () {56 vars in 1103 bytes} [Wed Jun 15 10:18:54 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:18:54.292112 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c49f09-1b8f-4d4d-a1cd-b193a996707f None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:18:54.327932 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 309/620] 10.222.0.49 () {58 vars in 1282 bytes} [Wed Jun 15 10:18:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 512 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 15 10:18:54.333801 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b496b1-0ddc-4523-a2f3-039ec7d1f83a None admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:18:54.393694 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:18:54.401524 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=100 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:18:54.468251 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c49f09-1b8f-4d4d-a1cd-b193a996707f None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:18:54.507135 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b496b1-0ddc-4523-a2f3-039ec7d1f83a None admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:18:54.515131 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e69703-bb87-4101-89e8-f1e322128b1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:18:54.516343 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e69703-bb87-4101-89e8-f1e322128b1c None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:54.516983 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e69703-bb87-4101-89e8-f1e322128b1c None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:54.517527 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e69703-bb87-4101-89e8-f1e322128b1c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:54.542485 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e69703-bb87-4101-89e8-f1e322128b1c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:18:54.544324 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e69703-bb87-4101-89e8-f1e322128b1c None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:18:54.549213 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc0a382-ba0e-4d7b-a21a-52b665837a05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:18:54.549213 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc0a382-ba0e-4d7b-a21a-52b665837a05 None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:54.549213 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc0a382-ba0e-4d7b-a21a-52b665837a05 None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:54.549213 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc0a382-ba0e-4d7b-a21a-52b665837a05 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:54.559522 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc0a382-ba0e-4d7b-a21a-52b665837a05 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:18:54.567197 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc0a382-ba0e-4d7b-a21a-52b665837a05 None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:18:54.578622 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application [None req-31e69703-bb87-4101-89e8-f1e322128b1c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 10:18:54.578622 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:18:54.578622 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:18:54.578622 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:18:54.578622 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:18:54.578622 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:18:54.578622 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:18:54.578622 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:18:54.578622 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:18:54.578622 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:18:54.578622 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:18:54.578622 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:18:54.578622 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 10:18:54.578622 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 10:18:54.578622 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 10:18:54.579615 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 10:18:54.579615 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:18:54.579615 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:18:54.579615 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:18:54.579615 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:18:54.579615 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:18:54.579615 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:18:54.579615 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:18:54.579615 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application with Lock( Jun 15 10:18:54.579615 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:18:54.579615 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:18:54.579615 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:18:54.579615 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:18:54.579615 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:18:54.579615 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:18:54.579615 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:18:54.579615 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:18:54.580541 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 10:18:54.580541 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 10:18:54.580541 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 10:18:54.580541 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Jun 15 10:18:54.580790 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application [None req-4fc0a382-ba0e-4d7b-a21a-52b665837a05 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 10:18:54.580790 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:18:54.580790 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:18:54.580790 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:18:54.580790 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:18:54.580790 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:18:54.580790 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:18:54.580790 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:18:54.580790 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:18:54.580790 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:18:54.580790 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:18:54.580790 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:18:54.580790 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 10:18:54.580790 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 10:18:54.580790 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 10:18:54.581587 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 10:18:54.581587 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:18:54.581587 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:18:54.581587 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:18:54.581587 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:18:54.581587 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:18:54.581587 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:18:54.581587 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:18:54.581587 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application with Lock( Jun 15 10:18:54.581587 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:18:54.581587 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:18:54.581587 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:18:54.581587 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:18:54.581587 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:18:54.581587 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:18:54.581587 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:18:54.581587 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:18:54.582490 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 10:18:54.582490 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 10:18:54.582490 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 10:18:54.582490 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application Jun 15 10:18:54.587197 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 310/621] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Jun 15 10:18:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:18:54.587197 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc0a382-ba0e-4d7b-a21a-52b665837a05 None admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:18:54.591117 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 312/622] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Jun 15 10:18:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 298 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:18:54.593691 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae1250a-588f-4863-8cab-cd5a28c5342c None None] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:54.594075 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae1250a-588f-4863-8cab-cd5a28c5342c None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:54.594445 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae1250a-588f-4863-8cab-cd5a28c5342c None None] PATH_INFO: `/` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:54.596137 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 313/623] 10.222.0.49 () {54 vars in 926 bytes} [Wed Jun 15 10:18:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:18:54.598309 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae1250a-588f-4863-8cab-cd5a28c5342c None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:18:54.689777 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=100 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:18:54.709636 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc0a382-ba0e-4d7b-a21a-52b665837a05 None admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:18:54.733994 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae1250a-588f-4863-8cab-cd5a28c5342c None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:18:54.737463 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:18:54.754397 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3d11b1-466f-477f-9721-860ce20bd9da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:18:54.755597 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3d11b1-466f-477f-9721-860ce20bd9da None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:54.755987 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3d11b1-466f-477f-9721-860ce20bd9da None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:54.756360 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3d11b1-466f-477f-9721-860ce20bd9da None admin] PATH_INFO: `/v3/domains` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:54.757178 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e3d11b1-466f-477f-9721-860ce20bd9da None admin] RBAC: Adding query filter params (name=Default) {{(pid=80119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:18:54.757581 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e3d11b1-466f-477f-9721-860ce20bd9da None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:18:54.759598 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e3d11b1-466f-477f-9721-860ce20bd9da None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:18:54.782878 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44788641-124b-41de-ac79-36b26d488a83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:18:54.784060 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44788641-124b-41de-ac79-36b26d488a83 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:54.784557 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44788641-124b-41de-ac79-36b26d488a83 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:54.785063 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44788641-124b-41de-ac79-36b26d488a83 None admin] PATH_INFO: `/v3/domains` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:54.785929 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 311/624] 10.222.0.49 () {56 vars in 1109 bytes} [Wed Jun 15 10:18:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:18:54.786289 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44788641-124b-41de-ac79-36b26d488a83 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:18:54.786813 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44788641-124b-41de-ac79-36b26d488a83 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:18:54.787495 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f11e38-d093-4b6a-9fd3-013c66efdc47 None None] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:54.787788 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f11e38-d093-4b6a-9fd3-013c66efdc47 None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:54.788067 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f11e38-d093-4b6a-9fd3-013c66efdc47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:54.795338 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44788641-124b-41de-ac79-36b26d488a83 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:18:54.809824 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.auth.core [None req-c4f11e38-d093-4b6a-9fd3-013c66efdc47 None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:18:54.811059 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-c4f11e38-d093-4b6a-9fd3-013c66efdc47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:18:54.812547 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 314/625] 10.222.0.49 () {56 vars in 1109 bytes} [Wed Jun 15 10:18:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 10:18:54.813759 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44788641-124b-41de-ac79-36b26d488a83 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:18:54.887998 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 312/626] 10.222.0.49 () {58 vars in 1023 bytes} [Wed Jun 15 10:18:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 10:18:54.888844 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f11e38-d093-4b6a-9fd3-013c66efdc47 None None] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:18:54.936040 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44788641-124b-41de-ac79-36b26d488a83 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:18:54.970690 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c834bdde-c1c9-460c-b48c-68c73b3b4cb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:18:54.972808 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c834bdde-c1c9-460c-b48c-68c73b3b4cb2 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:54.972808 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c834bdde-c1c9-460c-b48c-68c73b3b4cb2 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:54.972808 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c834bdde-c1c9-460c-b48c-68c73b3b4cb2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:54.976562 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=100 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:18:54.980953 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c834bdde-c1c9-460c-b48c-68c73b3b4cb2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:18:54.982423 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c834bdde-c1c9-460c-b48c-68c73b3b4cb2 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:18:54.996590 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application [None req-c834bdde-c1c9-460c-b48c-68c73b3b4cb2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 10:18:54.996590 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:18:54.996590 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:18:54.996590 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:18:54.996590 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:18:54.996590 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:18:54.996590 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:18:54.996590 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:18:54.996590 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:18:54.996590 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:18:54.996590 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:18:54.996590 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:18:54.996590 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 15 10:18:54.996590 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 15 10:18:54.996590 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 15 10:18:54.997504 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 15 10:18:54.997504 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:18:54.997504 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:18:54.997504 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:18:54.997504 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:18:54.997504 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:18:54.997504 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:18:54.997504 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:18:54.997504 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application with Lock( Jun 15 10:18:54.997504 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:18:54.997504 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:18:54.997504 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:18:54.997504 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:18:54.997504 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:18:54.997504 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:18:54.997504 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:18:54.997504 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:18:54.998443 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 15 10:18:54.998443 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 15 10:18:54.998443 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 15 10:18:54.998443 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 15 10:18:54.998443 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 15 10:18:54.998443 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 15 10:18:54.998443 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 10:18:54.998443 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Jun 15 10:18:54.998443 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 315/627] 10.222.0.49 () {56 vars in 1111 bytes} [Wed Jun 15 10:18:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:18:54.999266 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875e94f0-cd3a-4942-9241-9373fda803dc None None] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:54.999266 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875e94f0-cd3a-4942-9241-9373fda803dc None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:54.999547 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875e94f0-cd3a-4942-9241-9373fda803dc None None] PATH_INFO: `/` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:55.000940 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 316/628] 10.222.0.49 () {54 vars in 926 bytes} [Wed Jun 15 10:18:54 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:18:55.005380 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875e94f0-cd3a-4942-9241-9373fda803dc None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:18:55.007923 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f11e38-d093-4b6a-9fd3-013c66efdc47 None None] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:18:55.032530 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:18:55.044321 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6d6ad8-8581-469e-a490-48eab1f60681 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:18:55.045488 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6d6ad8-8581-469e-a490-48eab1f60681 None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:55.045488 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6d6ad8-8581-469e-a490-48eab1f60681 None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:55.045666 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6d6ad8-8581-469e-a490-48eab1f60681 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:55.053313 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af6d6ad8-8581-469e-a490-48eab1f60681 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:18:55.055163 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af6d6ad8-8581-469e-a490-48eab1f60681 None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:18:55.070496 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application [None req-af6d6ad8-8581-469e-a490-48eab1f60681 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 15 10:18:55.070496 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:18:55.070496 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:18:55.070496 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:18:55.070496 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:18:55.070496 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:18:55.070496 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:18:55.070496 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:18:55.070496 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:18:55.070496 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:18:55.070496 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:18:55.070496 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:18:55.070496 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 15 10:18:55.070496 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 15 10:18:55.070496 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 15 10:18:55.070496 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 15 10:18:55.071494 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:18:55.071494 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:18:55.071494 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 15 10:18:55.071494 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 10:18:55.071494 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 15 10:18:55.071494 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 10:18:55.071494 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:18:55.071494 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:18:55.071494 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:18:55.071494 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:18:55.071494 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:18:55.071494 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application with Lock( Jun 15 10:18:55.071494 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:18:55.071494 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:18:55.071494 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:18:55.071494 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:18:55.071494 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:18:55.071494 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:18:55.072540 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:18:55.072540 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:18:55.072540 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 15 10:18:55.072540 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 15 10:18:55.072540 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 15 10:18:55.072540 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 15 10:18:55.072540 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 15 10:18:55.072540 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 15 10:18:55.072540 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 15 10:18:55.072540 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application Jun 15 10:18:55.072540 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 313/629] 10.222.0.49 () {56 vars in 1096 bytes} [Wed Jun 15 10:18:54 2022] GET /identity/v3/users/swift => generated 83 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:18:55.073144 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6d6ad8-8581-469e-a490-48eab1f60681 None admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:18:55.118478 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875e94f0-cd3a-4942-9241-9373fda803dc None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:18:55.152866 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac8e80e-e4b5-486f-80aa-ba01cd831f0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:18:55.153674 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac8e80e-e4b5-486f-80aa-ba01cd831f0d None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:55.153957 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac8e80e-e4b5-486f-80aa-ba01cd831f0d None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:55.154230 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac8e80e-e4b5-486f-80aa-ba01cd831f0d None admin] PATH_INFO: `/v3/roles/service` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:55.164684 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac8e80e-e4b5-486f-80aa-ba01cd831f0d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 10:18:55.172980 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application [None req-2ac8e80e-e4b5-486f-80aa-ba01cd831f0d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 15 10:18:55.172980 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:18:55.172980 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:18:55.172980 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:18:55.172980 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:18:55.172980 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:18:55.172980 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:18:55.172980 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:18:55.172980 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:18:55.172980 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:18:55.172980 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:18:55.172980 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:18:55.172980 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 10:18:55.172980 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 10:18:55.172980 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 10:18:55.172980 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 10:18:55.173921 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 10:18:55.173921 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 10:18:55.173921 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 10:18:55.173921 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 10:18:55.173921 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:18:55.173921 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:18:55.173921 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:18:55.173921 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:18:55.173921 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:18:55.173921 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:18:55.173921 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:18:55.173921 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application with Lock( Jun 15 10:18:55.173921 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:18:55.173921 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:18:55.173921 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:18:55.173921 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:18:55.174809 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:18:55.174809 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:18:55.174809 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:18:55.174809 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:18:55.174809 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 10:18:55.174809 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 10:18:55.174809 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 10:18:55.174809 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 10:18:55.174809 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 10:18:55.174809 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 10:18:55.174809 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 15 10:18:55.174809 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Jun 15 10:18:55.179215 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 317/630] 10.222.0.49 () {56 vars in 1102 bytes} [Wed Jun 15 10:18:55 2022] GET /identity/v3/roles/service => generated 85 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:18:55.180503 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3feadb9-bad2-49e3-91d6-f5e7b6c0fbcb None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:55.180796 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3feadb9-bad2-49e3-91d6-f5e7b6c0fbcb None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:55.181078 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3feadb9-bad2-49e3-91d6-f5e7b6c0fbcb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:55.183806 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6d6ad8-8581-469e-a490-48eab1f60681 None admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:18:55.197876 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-c3feadb9-bad2-49e3-91d6-f5e7b6c0fbcb None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:18:55.199148 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-c3feadb9-bad2-49e3-91d6-f5e7b6c0fbcb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:18:55.224241 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924ef00b-0e2e-4207-bcbd-82e0681ff04d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:18:55.225251 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924ef00b-0e2e-4207-bcbd-82e0681ff04d None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:55.225544 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924ef00b-0e2e-4207-bcbd-82e0681ff04d None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:55.225828 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924ef00b-0e2e-4207-bcbd-82e0681ff04d None admin] PATH_INFO: `/v3/projects` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:55.226614 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-924ef00b-0e2e-4207-bcbd-82e0681ff04d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:18:55.226926 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-924ef00b-0e2e-4207-bcbd-82e0681ff04d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:18:55.228474 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-924ef00b-0e2e-4207-bcbd-82e0681ff04d None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:18:55.231870 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=100 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:18:55.248624 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 314/631] 10.222.0.49 () {56 vars in 1148 bytes} [Wed Jun 15 10:18:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:18:55.248624 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924ef00b-0e2e-4207-bcbd-82e0681ff04d None admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:18:55.279397 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 318/632] 10.222.0.49 () {58 vars in 1023 bytes} [Wed Jun 15 10:18:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 10:18:55.280138 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3feadb9-bad2-49e3-91d6-f5e7b6c0fbcb None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:18:55.299846 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:18:55.357727 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924ef00b-0e2e-4207-bcbd-82e0681ff04d None admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:18:55.387589 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3feadb9-bad2-49e3-91d6-f5e7b6c0fbcb None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:18:55.393255 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5884905c-5739-460d-a662-716191c4ff81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:18:55.394169 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5884905c-5739-460d-a662-716191c4ff81 None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:55.394681 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5884905c-5739-460d-a662-716191c4ff81 None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:55.395197 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5884905c-5739-460d-a662-716191c4ff81 None admin] PATH_INFO: `/v3/users` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:55.396107 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5884905c-5739-460d-a662-716191c4ff81 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:18:55.396451 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5884905c-5739-460d-a662-716191c4ff81 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:18:55.397878 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5884905c-5739-460d-a662-716191c4ff81 None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:18:55.421540 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 315/633] 10.222.0.49 () {56 vars in 1135 bytes} [Wed Jun 15 10:18:55 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:18:55.421540 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5884905c-5739-460d-a662-716191c4ff81 None admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:18:55.428729 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b674bd8-e451-4d69-8783-82ab8b78a713 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:18:55.429415 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b674bd8-e451-4d69-8783-82ab8b78a713 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:55.429711 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b674bd8-e451-4d69-8783-82ab8b78a713 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:55.429983 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b674bd8-e451-4d69-8783-82ab8b78a713 None admin] PATH_INFO: `/v3/roles` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:55.430741 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b674bd8-e451-4d69-8783-82ab8b78a713 None admin] RBAC: Adding query filter params (name=service) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:18:55.431047 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b674bd8-e451-4d69-8783-82ab8b78a713 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:18:55.432453 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b674bd8-e451-4d69-8783-82ab8b78a713 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:18:55.441764 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 319/634] 10.222.0.49 () {56 vars in 1103 bytes} [Wed Jun 15 10:18:55 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:18:55.442568 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b674bd8-e451-4d69-8783-82ab8b78a713 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:18:55.482046 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=100 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:18:55.529432 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:18:55.534972 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5884905c-5739-460d-a662-716191c4ff81 None admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:18:55.555357 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b674bd8-e451-4d69-8783-82ab8b78a713 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:18:55.569923 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db6be47-b108-4465-ba0c-eb39ff79f5f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:18:55.570655 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db6be47-b108-4465-ba0c-eb39ff79f5f8 None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:55.571009 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db6be47-b108-4465-ba0c-eb39ff79f5f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:55.571324 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db6be47-b108-4465-ba0c-eb39ff79f5f8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:55.572153 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db6be47-b108-4465-ba0c-eb39ff79f5f8 None admin] RBAC: Adding query filter params (role.id=16e76452bb5d4ce68a119ad3c9bde41c, scope.project.id=559b3e713f5844b6ae37151d56dd419f, user.id=5ac54528370d4747b6e8777fba9d0c82) {{(pid=80119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:18:55.572464 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db6be47-b108-4465-ba0c-eb39ff79f5f8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:18:55.573918 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db6be47-b108-4465-ba0c-eb39ff79f5f8 None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:18:55.579674 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 316/635] 10.222.0.49 () {56 vars in 1374 bytes} [Wed Jun 15 10:18:55 2022] GET /identity/v3/role_assignments?user.id=5ac54528370d4747b6e8777fba9d0c82&scope.project.id=559b3e713f5844b6ae37151d56dd419f&role.id=16e76452bb5d4ce68a119ad3c9bde41c => generated 608 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:18:55.580445 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db6be47-b108-4465-ba0c-eb39ff79f5f8 None admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:18:55.590363 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f561fc-a95c-468a-8eb3-84372e5fd5c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:18:55.591237 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f561fc-a95c-468a-8eb3-84372e5fd5c3 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:55.591609 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f561fc-a95c-468a-8eb3-84372e5fd5c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:55.591889 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f561fc-a95c-468a-8eb3-84372e5fd5c3 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:55.605592 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f561fc-a95c-468a-8eb3-84372e5fd5c3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 10:18:55.613569 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application [None req-e2f561fc-a95c-468a-8eb3-84372e5fd5c3 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 15 10:18:55.613569 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:18:55.613569 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:18:55.613569 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:18:55.613569 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:18:55.613569 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:18:55.613569 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:18:55.613569 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:18:55.613569 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:18:55.613569 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:18:55.613569 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:18:55.613569 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:18:55.613569 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 10:18:55.613569 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 10:18:55.613569 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 10:18:55.613569 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 10:18:55.614804 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 10:18:55.614804 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 10:18:55.614804 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 10:18:55.614804 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 10:18:55.614804 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:18:55.614804 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:18:55.614804 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:18:55.614804 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:18:55.614804 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:18:55.614804 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:18:55.614804 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:18:55.614804 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application with Lock( Jun 15 10:18:55.614804 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:18:55.614804 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:18:55.614804 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:18:55.614804 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:18:55.615838 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:18:55.615838 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:18:55.615838 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:18:55.615838 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:18:55.615838 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 10:18:55.615838 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 10:18:55.615838 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 10:18:55.615838 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 10:18:55.615838 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 10:18:55.615838 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 10:18:55.615838 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 15 10:18:55.615838 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Jun 15 10:18:55.615838 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 320/636] 10.222.0.49 () {56 vars in 1102 bytes} [Wed Jun 15 10:18:55 2022] GET /identity/v3/roles/service => generated 85 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:18:55.615838 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f561fc-a95c-468a-8eb3-84372e5fd5c3 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:18:55.701931 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db6be47-b108-4465-ba0c-eb39ff79f5f8 None admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:18:55.721223 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=100 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:18:55.739879 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f561fc-a95c-468a-8eb3-84372e5fd5c3 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:18:55.743351 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8144781-0938-4b39-99e1-ce7221b4ec79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:18:55.744161 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8144781-0938-4b39-99e1-ce7221b4ec79 None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:55.744441 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8144781-0938-4b39-99e1-ce7221b4ec79 None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:55.744714 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8144781-0938-4b39-99e1-ce7221b4ec79 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:55.756462 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8144781-0938-4b39-99e1-ce7221b4ec79 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:18:55.757948 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8144781-0938-4b39-99e1-ce7221b4ec79 None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:18:55.778042 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application [None req-c8144781-0938-4b39-99e1-ce7221b4ec79 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 10:18:55.778042 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:18:55.778042 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:18:55.778042 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:18:55.778042 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:18:55.778042 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:18:55.778042 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:18:55.778042 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:18:55.778042 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:18:55.778042 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:18:55.778042 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:18:55.778042 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:18:55.778042 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 10:18:55.778042 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 10:18:55.778042 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 10:18:55.778988 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 10:18:55.778988 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:18:55.778988 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:18:55.778988 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:18:55.778988 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:18:55.778988 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:18:55.778988 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:18:55.778988 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:18:55.778988 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application with Lock( Jun 15 10:18:55.778988 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:18:55.778988 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:18:55.778988 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:18:55.778988 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:18:55.778988 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:18:55.778988 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:18:55.778988 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:18:55.778988 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:18:55.780357 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 10:18:55.780357 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 10:18:55.780357 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 10:18:55.780357 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application Jun 15 10:18:55.780357 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 317/637] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Jun 15 10:18:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:18:55.780939 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a440dd4f-0b6b-4bb3-a971-8e65b8309c60 None None] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:55.780939 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a440dd4f-0b6b-4bb3-a971-8e65b8309c60 None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:55.781255 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a440dd4f-0b6b-4bb3-a971-8e65b8309c60 None None] PATH_INFO: `/` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:55.782293 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998d18ad-39d8-4e85-beed-eb785b804fec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:18:55.787073 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998d18ad-39d8-4e85-beed-eb785b804fec None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:55.787297 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 318/638] 10.222.0.49 () {54 vars in 926 bytes} [Wed Jun 15 10:18:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:18:55.787377 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998d18ad-39d8-4e85-beed-eb785b804fec None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:55.787377 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998d18ad-39d8-4e85-beed-eb785b804fec None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:55.787514 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a440dd4f-0b6b-4bb3-a971-8e65b8309c60 None None] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:18:55.797056 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:18:55.804215 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-998d18ad-39d8-4e85-beed-eb785b804fec None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:18:55.806358 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-998d18ad-39d8-4e85-beed-eb785b804fec None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:18:55.826218 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application [None req-998d18ad-39d8-4e85-beed-eb785b804fec None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 10:18:55.826218 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:18:55.826218 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:18:55.826218 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:18:55.826218 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:18:55.826218 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:18:55.826218 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:18:55.826218 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:18:55.826218 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:18:55.826218 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:18:55.826218 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:18:55.826218 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:18:55.826218 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 10:18:55.826218 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 10:18:55.826218 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 10:18:55.827288 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 10:18:55.827288 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:18:55.827288 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:18:55.827288 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:18:55.827288 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:18:55.827288 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:18:55.827288 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:18:55.827288 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:18:55.827288 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application with Lock( Jun 15 10:18:55.827288 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:18:55.827288 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:18:55.827288 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:18:55.827288 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:18:55.827288 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:18:55.827288 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:18:55.827288 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:18:55.827288 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:18:55.828409 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 10:18:55.828409 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 10:18:55.828409 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 10:18:55.828409 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Jun 15 10:18:55.828409 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 321/639] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Jun 15 10:18:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:18:55.828409 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998d18ad-39d8-4e85-beed-eb785b804fec None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:18:55.903910 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a440dd4f-0b6b-4bb3-a971-8e65b8309c60 None None] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:18:55.936571 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998d18ad-39d8-4e85-beed-eb785b804fec None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:18:55.943287 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b256971b-6f1c-4a03-b5f6-72a61680e063 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:18:55.944366 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b256971b-6f1c-4a03-b5f6-72a61680e063 None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:55.944656 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b256971b-6f1c-4a03-b5f6-72a61680e063 None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:55.944933 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b256971b-6f1c-4a03-b5f6-72a61680e063 None admin] PATH_INFO: `/v3/roles` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:55.946035 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b256971b-6f1c-4a03-b5f6-72a61680e063 None admin] RBAC: Adding query filter params (name=service) {{(pid=80119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:18:55.946347 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b256971b-6f1c-4a03-b5f6-72a61680e063 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:18:55.948065 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b256971b-6f1c-4a03-b5f6-72a61680e063 None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:18:55.961344 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 319/640] 10.222.0.49 () {56 vars in 1103 bytes} [Wed Jun 15 10:18:55 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:18:55.962695 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e708b64-8290-4e6c-b6da-08e64f5af1de None None] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:55.962990 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e708b64-8290-4e6c-b6da-08e64f5af1de None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:55.963301 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e708b64-8290-4e6c-b6da-08e64f5af1de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:55.972582 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3c797a-9a86-418f-88d6-adad8a6825d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:18:55.973248 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3c797a-9a86-418f-88d6-adad8a6825d5 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:55.973530 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3c797a-9a86-418f-88d6-adad8a6825d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:55.973801 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3c797a-9a86-418f-88d6-adad8a6825d5 None admin] PATH_INFO: `/v3/domains` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:55.974999 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a3c797a-9a86-418f-88d6-adad8a6825d5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:18:55.975331 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a3c797a-9a86-418f-88d6-adad8a6825d5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:18:55.976699 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a3c797a-9a86-418f-88d6-adad8a6825d5 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:18:55.980996 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.auth.core [None req-6e708b64-8290-4e6c-b6da-08e64f5af1de None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:18:55.982213 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-6e708b64-8290-4e6c-b6da-08e64f5af1de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:18:55.993261 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 322/641] 10.222.0.49 () {56 vars in 1109 bytes} [Wed Jun 15 10:18:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:18:55.993261 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3c797a-9a86-418f-88d6-adad8a6825d5 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:18:56.014681 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=100 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:18:56.042063 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:18:56.061528 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 320/642] 10.222.0.49 () {58 vars in 1023 bytes} [Wed Jun 15 10:18:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 10:18:56.063390 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e708b64-8290-4e6c-b6da-08e64f5af1de None None] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:18:56.100507 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3c797a-9a86-418f-88d6-adad8a6825d5 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:18:56.139808 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8eac858-0340-43ec-8133-c9f6b6ef33f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:18:56.140488 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8eac858-0340-43ec-8133-c9f6b6ef33f6 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:56.141099 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8eac858-0340-43ec-8133-c9f6b6ef33f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:56.141099 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8eac858-0340-43ec-8133-c9f6b6ef33f6 None admin] PATH_INFO: `/v3/domains` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:56.141760 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8eac858-0340-43ec-8133-c9f6b6ef33f6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:18:56.142066 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8eac858-0340-43ec-8133-c9f6b6ef33f6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:18:56.143472 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8eac858-0340-43ec-8133-c9f6b6ef33f6 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:18:56.159630 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 323/643] 10.222.0.49 () {56 vars in 1109 bytes} [Wed Jun 15 10:18:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:18:56.159630 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8eac858-0340-43ec-8133-c9f6b6ef33f6 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:18:56.179802 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e708b64-8290-4e6c-b6da-08e64f5af1de None None] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:18:56.216214 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aacf99e3-7713-47e0-b897-7bf594074593 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:18:56.217021 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacf99e3-7713-47e0-b897-7bf594074593 None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:56.217309 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacf99e3-7713-47e0-b897-7bf594074593 None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:56.217589 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacf99e3-7713-47e0-b897-7bf594074593 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:56.228893 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aacf99e3-7713-47e0-b897-7bf594074593 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:18:56.230393 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aacf99e3-7713-47e0-b897-7bf594074593 None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:18:56.249053 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application [None req-aacf99e3-7713-47e0-b897-7bf594074593 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 10:18:56.249053 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:18:56.249053 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:18:56.249053 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:18:56.249053 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:18:56.249053 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:18:56.249053 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:18:56.249053 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:18:56.249053 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:18:56.249053 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:18:56.249053 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:18:56.249053 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:18:56.249053 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 10:18:56.249053 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 10:18:56.249053 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 10:18:56.250213 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 10:18:56.250213 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:18:56.250213 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:18:56.250213 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:18:56.250213 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:18:56.250213 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:18:56.250213 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:18:56.250213 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:18:56.250213 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application with Lock( Jun 15 10:18:56.250213 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:18:56.250213 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:18:56.250213 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:18:56.250213 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:18:56.250213 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:18:56.250213 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:18:56.250213 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:18:56.250213 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:18:56.251391 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 10:18:56.251391 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 10:18:56.251391 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 10:18:56.251391 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application Jun 15 10:18:56.251391 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 321/644] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Jun 15 10:18:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:18:56.251391 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aacf99e3-7713-47e0-b897-7bf594074593 None admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:18:56.266998 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=100 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:18:56.277039 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8eac858-0340-43ec-8133-c9f6b6ef33f6 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:18:56.286768 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:18:56.314496 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82dd6404-3eb5-4f64-84df-8e1626bd1607 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:18:56.315466 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82dd6404-3eb5-4f64-84df-8e1626bd1607 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:56.315848 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82dd6404-3eb5-4f64-84df-8e1626bd1607 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:56.316248 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82dd6404-3eb5-4f64-84df-8e1626bd1607 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:56.323207 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82dd6404-3eb5-4f64-84df-8e1626bd1607 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:18:56.325251 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82dd6404-3eb5-4f64-84df-8e1626bd1607 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:18:56.339369 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application [None req-82dd6404-3eb5-4f64-84df-8e1626bd1607 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 10:18:56.339369 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:18:56.339369 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:18:56.339369 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:18:56.339369 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:18:56.339369 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:18:56.339369 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:18:56.339369 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:18:56.339369 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:18:56.339369 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:18:56.339369 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:18:56.339369 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:18:56.339369 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 15 10:18:56.339369 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 15 10:18:56.339369 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 15 10:18:56.340494 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 15 10:18:56.340494 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:18:56.340494 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:18:56.340494 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:18:56.340494 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:18:56.340494 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:18:56.340494 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:18:56.340494 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:18:56.340494 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application with Lock( Jun 15 10:18:56.340494 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:18:56.340494 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:18:56.340494 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:18:56.340494 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:18:56.340494 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:18:56.340494 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:18:56.340494 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:18:56.340494 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:18:56.341732 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 15 10:18:56.341732 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 15 10:18:56.341732 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 15 10:18:56.341732 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 15 10:18:56.341732 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 15 10:18:56.341732 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 15 10:18:56.341732 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 10:18:56.341732 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Jun 15 10:18:56.341732 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 324/645] 10.222.0.49 () {56 vars in 1111 bytes} [Wed Jun 15 10:18:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:18:56.346669 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82dd6404-3eb5-4f64-84df-8e1626bd1607 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:18:56.360742 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aacf99e3-7713-47e0-b897-7bf594074593 None admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:18:56.398491 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9e480e-a179-41c5-bafa-95a45bd7ab6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:18:56.399316 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9e480e-a179-41c5-bafa-95a45bd7ab6d None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:56.399601 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9e480e-a179-41c5-bafa-95a45bd7ab6d None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:56.399878 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9e480e-a179-41c5-bafa-95a45bd7ab6d None admin] PATH_INFO: `/v3/roles/service` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:56.412288 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab9e480e-a179-41c5-bafa-95a45bd7ab6d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 10:18:56.420764 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application [None req-ab9e480e-a179-41c5-bafa-95a45bd7ab6d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 15 10:18:56.420764 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:18:56.420764 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:18:56.420764 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:18:56.420764 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:18:56.420764 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:18:56.420764 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:18:56.420764 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:18:56.420764 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:18:56.420764 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:18:56.420764 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:18:56.420764 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:18:56.420764 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 10:18:56.420764 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 10:18:56.420764 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 10:18:56.420764 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 10:18:56.421721 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 10:18:56.421721 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 10:18:56.421721 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 10:18:56.421721 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 10:18:56.421721 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:18:56.421721 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:18:56.421721 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:18:56.421721 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:18:56.421721 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:18:56.421721 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:18:56.421721 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:18:56.421721 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application with Lock( Jun 15 10:18:56.421721 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:18:56.421721 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:18:56.421721 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:18:56.421721 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:18:56.423145 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:18:56.423145 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:18:56.423145 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:18:56.423145 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:18:56.423145 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 10:18:56.423145 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 10:18:56.423145 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 10:18:56.423145 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 10:18:56.423145 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 10:18:56.423145 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 10:18:56.423145 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 15 10:18:56.423145 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application Jun 15 10:18:56.423145 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 322/646] 10.222.0.49 () {56 vars in 1102 bytes} [Wed Jun 15 10:18:56 2022] GET /identity/v3/roles/service => generated 85 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:18:56.423145 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9e480e-a179-41c5-bafa-95a45bd7ab6d None admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:18:56.455320 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82dd6404-3eb5-4f64-84df-8e1626bd1607 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:18:56.490295 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9afbafb1-7678-41d8-999f-2802f54d82a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:18:56.491121 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afbafb1-7678-41d8-999f-2802f54d82a7 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:56.491423 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afbafb1-7678-41d8-999f-2802f54d82a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:56.491701 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afbafb1-7678-41d8-999f-2802f54d82a7 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:56.499498 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9afbafb1-7678-41d8-999f-2802f54d82a7 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:18:56.500937 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9afbafb1-7678-41d8-999f-2802f54d82a7 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:18:56.506330 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=100 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:18:56.515928 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application [None req-9afbafb1-7678-41d8-999f-2802f54d82a7 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jun 15 10:18:56.515928 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:18:56.515928 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:18:56.515928 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:18:56.515928 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:18:56.515928 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:18:56.515928 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:18:56.515928 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:18:56.515928 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:18:56.515928 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:18:56.515928 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:18:56.515928 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:18:56.515928 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 15 10:18:56.515928 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 15 10:18:56.515928 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 15 10:18:56.515928 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 15 10:18:56.517082 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:18:56.517082 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:18:56.517082 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 15 10:18:56.517082 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 10:18:56.517082 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 15 10:18:56.517082 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 10:18:56.517082 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:18:56.517082 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:18:56.517082 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:18:56.517082 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:18:56.517082 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:18:56.517082 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application with Lock( Jun 15 10:18:56.517082 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:18:56.517082 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:18:56.517082 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:18:56.517082 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:18:56.517082 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:18:56.517082 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:18:56.518026 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:18:56.518026 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:18:56.518026 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 15 10:18:56.518026 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 15 10:18:56.518026 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 15 10:18:56.518026 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 15 10:18:56.518026 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 15 10:18:56.518026 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 15 10:18:56.518026 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jun 15 10:18:56.518026 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Jun 15 10:18:56.518026 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 325/647] 10.222.0.49 () {56 vars in 1099 bytes} [Wed Jun 15 10:18:56 2022] GET /identity/v3/users/glance => generated 84 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:18:56.518026 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9afbafb1-7678-41d8-999f-2802f54d82a7 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:18:56.536870 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9e480e-a179-41c5-bafa-95a45bd7ab6d None admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:18:56.548256 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:18:56.573790 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7da13b9-7f7f-4a9b-b2b0-d6e387272c62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:18:56.574390 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7da13b9-7f7f-4a9b-b2b0-d6e387272c62 None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:56.574667 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7da13b9-7f7f-4a9b-b2b0-d6e387272c62 None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:56.574943 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7da13b9-7f7f-4a9b-b2b0-d6e387272c62 None admin] PATH_INFO: `/v3/domains` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:56.576109 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7da13b9-7f7f-4a9b-b2b0-d6e387272c62 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:18:56.576568 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7da13b9-7f7f-4a9b-b2b0-d6e387272c62 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:18:56.577940 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7da13b9-7f7f-4a9b-b2b0-d6e387272c62 None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:18:56.590126 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 323/648] 10.222.0.49 () {56 vars in 1109 bytes} [Wed Jun 15 10:18:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:18:56.591215 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7da13b9-7f7f-4a9b-b2b0-d6e387272c62 None admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:18:56.631867 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9afbafb1-7678-41d8-999f-2802f54d82a7 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:18:56.666978 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4332ec-e822-4857-bd21-af835519f6d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:18:56.668238 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4332ec-e822-4857-bd21-af835519f6d9 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:56.668512 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4332ec-e822-4857-bd21-af835519f6d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:56.668794 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4332ec-e822-4857-bd21-af835519f6d9 None admin] PATH_INFO: `/v3/projects` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:56.669583 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f4332ec-e822-4857-bd21-af835519f6d9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:18:56.669893 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f4332ec-e822-4857-bd21-af835519f6d9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:18:56.671355 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f4332ec-e822-4857-bd21-af835519f6d9 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:18:56.684145 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 326/649] 10.222.0.49 () {56 vars in 1148 bytes} [Wed Jun 15 10:18:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:18:56.689402 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4332ec-e822-4857-bd21-af835519f6d9 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:18:56.694153 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7da13b9-7f7f-4a9b-b2b0-d6e387272c62 None admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:18:56.735986 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0932ca-a802-4c23-bcac-445916c93a09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:18:56.736690 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0932ca-a802-4c23-bcac-445916c93a09 None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:56.736976 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0932ca-a802-4c23-bcac-445916c93a09 None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:56.737257 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0932ca-a802-4c23-bcac-445916c93a09 None admin] PATH_INFO: `/v3/roles` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:56.738020 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d0932ca-a802-4c23-bcac-445916c93a09 None admin] RBAC: Adding query filter params (name=service) {{(pid=80119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:18:56.738334 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d0932ca-a802-4c23-bcac-445916c93a09 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:18:56.739787 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d0932ca-a802-4c23-bcac-445916c93a09 None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:18:56.741553 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=100 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:18:56.755408 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 324/650] 10.222.0.49 () {56 vars in 1103 bytes} [Wed Jun 15 10:18:56 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:18:56.756396 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0932ca-a802-4c23-bcac-445916c93a09 None admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:18:56.825765 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:18:56.841910 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4332ec-e822-4857-bd21-af835519f6d9 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:18:56.878721 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c46eae-4f52-406d-a871-134c33179cac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:18:56.879510 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c46eae-4f52-406d-a871-134c33179cac None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:56.879858 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c46eae-4f52-406d-a871-134c33179cac None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:56.880141 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c46eae-4f52-406d-a871-134c33179cac None admin] PATH_INFO: `/v3/users` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:56.880976 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c46eae-4f52-406d-a871-134c33179cac None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:18:56.881303 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c46eae-4f52-406d-a871-134c33179cac None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:18:56.882715 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c46eae-4f52-406d-a871-134c33179cac None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:18:56.890762 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0932ca-a802-4c23-bcac-445916c93a09 None admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:18:56.901093 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 327/651] 10.222.0.49 () {56 vars in 1137 bytes} [Wed Jun 15 10:18:56 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:18:56.902406 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c46eae-4f52-406d-a871-134c33179cac None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:18:56.946120 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3fcc2d-e1be-42bf-ba43-5feeb1124d71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:18:56.947008 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3fcc2d-e1be-42bf-ba43-5feeb1124d71 None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:56.947244 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3fcc2d-e1be-42bf-ba43-5feeb1124d71 None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:56.947520 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3fcc2d-e1be-42bf-ba43-5feeb1124d71 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:56.961536 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d3fcc2d-e1be-42bf-ba43-5feeb1124d71 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:18:56.962620 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d3fcc2d-e1be-42bf-ba43-5feeb1124d71 None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:18:56.981523 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application [None req-3d3fcc2d-e1be-42bf-ba43-5feeb1124d71 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 15 10:18:56.981523 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:18:56.981523 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:18:56.981523 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:18:56.981523 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:18:56.981523 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:18:56.981523 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:18:56.981523 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:18:56.981523 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:18:56.981523 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:18:56.981523 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:18:56.981523 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:18:56.981523 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 15 10:18:56.981523 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 15 10:18:56.981523 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 15 10:18:56.981523 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 15 10:18:56.982531 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:18:56.982531 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:18:56.982531 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 15 10:18:56.982531 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 10:18:56.982531 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 15 10:18:56.982531 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 10:18:56.982531 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:18:56.982531 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:18:56.982531 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:18:56.982531 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:18:56.982531 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:18:56.982531 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application with Lock( Jun 15 10:18:56.982531 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:18:56.982531 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:18:56.982531 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:18:56.982531 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:18:56.982531 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:18:56.982531 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:18:56.983520 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:18:56.983520 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:18:56.983520 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 15 10:18:56.983520 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 15 10:18:56.983520 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 15 10:18:56.983520 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 15 10:18:56.983520 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 15 10:18:56.983520 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 15 10:18:56.983520 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 15 10:18:56.983520 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application Jun 15 10:18:56.989078 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 325/652] 10.222.0.49 () {56 vars in 1099 bytes} [Wed Jun 15 10:18:56 2022] GET /identity/v3/users/cinder => generated 84 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:18:56.990115 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3fcc2d-e1be-42bf-ba43-5feeb1124d71 None admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:18:57.016402 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c46eae-4f52-406d-a871-134c33179cac None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:18:57.048960 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=100 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:18:57.074187 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3652c6a-db98-4bea-bce4-f533f6ed2d46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:18:57.075279 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3652c6a-db98-4bea-bce4-f533f6ed2d46 None admin] REQUEST_METHOD: `PUT` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:57.075630 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3652c6a-db98-4bea-bce4-f533f6ed2d46 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:57.076023 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3652c6a-db98-4bea-bce4-f533f6ed2d46 None admin] PATH_INFO: `/v3/projects/559b3e713f5844b6ae37151d56dd419f/users/3a69f99210444f9696e8b22615fb4a41/roles/16e76452bb5d4ce68a119ad3c9bde41c` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:57.091995 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3652c6a-db98-4bea-bce4-f533f6ed2d46 None admin] RBAC: Authorizing `identity:create_grant(project_id=559b3e713f5844b6ae37151d56dd419f, user_id=3a69f99210444f9696e8b22615fb4a41, role_id=16e76452bb5d4ce68a119ad3c9bde41c)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:18:57.093944 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3652c6a-db98-4bea-bce4-f533f6ed2d46 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:18:57.096258 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3fcc2d-e1be-42bf-ba43-5feeb1124d71 None admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:18:57.097464 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:18:57.124059 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 328/653] 10.222.0.49 () {58 vars in 1439 bytes} [Wed Jun 15 10:18:56 2022] PUT /identity/v3/projects/559b3e713f5844b6ae37151d56dd419f/users/3a69f99210444f9696e8b22615fb4a41/roles/16e76452bb5d4ce68a119ad3c9bde41c => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:18:57.125213 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3652c6a-db98-4bea-bce4-f533f6ed2d46 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:18:57.144527 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0f21bd-ed0d-4088-935c-ff5a9121abc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:18:57.146303 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0f21bd-ed0d-4088-935c-ff5a9121abc9 None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:57.146763 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0f21bd-ed0d-4088-935c-ff5a9121abc9 None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:57.147255 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0f21bd-ed0d-4088-935c-ff5a9121abc9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:57.161966 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a0f21bd-ed0d-4088-935c-ff5a9121abc9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:18:57.163998 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a0f21bd-ed0d-4088-935c-ff5a9121abc9 None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:18:57.185634 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application [None req-4a0f21bd-ed0d-4088-935c-ff5a9121abc9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 10:18:57.185634 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:18:57.185634 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:18:57.185634 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:18:57.185634 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:18:57.185634 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:18:57.185634 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:18:57.185634 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:18:57.185634 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:18:57.185634 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:18:57.185634 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:18:57.185634 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:18:57.185634 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 10:18:57.185634 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 10:18:57.185634 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 10:18:57.186822 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 10:18:57.186822 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:18:57.186822 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:18:57.186822 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:18:57.186822 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:18:57.186822 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:18:57.186822 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:18:57.186822 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:18:57.186822 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application with Lock( Jun 15 10:18:57.186822 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:18:57.186822 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:18:57.186822 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:18:57.186822 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:18:57.186822 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:18:57.186822 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:18:57.186822 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:18:57.186822 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:18:57.187984 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 10:18:57.187984 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 10:18:57.187984 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 10:18:57.187984 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application Jun 15 10:18:57.194907 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 326/654] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Jun 15 10:18:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:18:57.196130 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0f21bd-ed0d-4088-935c-ff5a9121abc9 None admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:18:57.297871 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3652c6a-db98-4bea-bce4-f533f6ed2d46 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:18:57.310007 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0f21bd-ed0d-4088-935c-ff5a9121abc9 None admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:18:57.319485 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=100 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:18:57.340729 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b184df-5df4-4f07-b204-60da71a85731 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:18:57.341514 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b184df-5df4-4f07-b204-60da71a85731 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:57.341951 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b184df-5df4-4f07-b204-60da71a85731 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:57.342400 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b184df-5df4-4f07-b204-60da71a85731 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:57.349957 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243731e7-568e-48f1-8f4f-c4137854c28b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:18:57.350753 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243731e7-568e-48f1-8f4f-c4137854c28b None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:57.351119 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243731e7-568e-48f1-8f4f-c4137854c28b None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:57.351412 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243731e7-568e-48f1-8f4f-c4137854c28b None admin] PATH_INFO: `/v3/users` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:57.352261 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243731e7-568e-48f1-8f4f-c4137854c28b None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:18:57.353443 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243731e7-568e-48f1-8f4f-c4137854c28b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:18:57.354548 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243731e7-568e-48f1-8f4f-c4137854c28b None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:18:57.377485 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 327/655] 10.222.0.49 () {56 vars in 1137 bytes} [Wed Jun 15 10:18:57 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:18:57.378436 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243731e7-568e-48f1-8f4f-c4137854c28b None admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:18:57.411563 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:18:57.419129 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b184df-5df4-4f07-b204-60da71a85731 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:18:57.420538 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b184df-5df4-4f07-b204-60da71a85731 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:18:57.480140 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243731e7-568e-48f1-8f4f-c4137854c28b None admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:18:57.515390 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52839a72-7be7-4310-ace0-0903e8716963 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:18:57.516061 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52839a72-7be7-4310-ace0-0903e8716963 None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:57.516376 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52839a72-7be7-4310-ace0-0903e8716963 None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:57.516650 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52839a72-7be7-4310-ace0-0903e8716963 None admin] PATH_INFO: `/v3/domains` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:57.517371 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52839a72-7be7-4310-ace0-0903e8716963 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:18:57.517678 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52839a72-7be7-4310-ace0-0903e8716963 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:18:57.519223 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52839a72-7be7-4310-ace0-0903e8716963 None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:18:57.530738 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 329/656] 10.222.0.49 () {58 vars in 1282 bytes} [Wed Jun 15 10:18:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 406 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 15 10:18:57.531783 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b184df-5df4-4f07-b204-60da71a85731 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:18:57.532853 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 328/657] 10.222.0.49 () {56 vars in 1109 bytes} [Wed Jun 15 10:18:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:18:57.539256 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52839a72-7be7-4310-ace0-0903e8716963 None admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:18:57.567366 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=100 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:18:57.635373 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:18:57.641606 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52839a72-7be7-4310-ace0-0903e8716963 None admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:18:57.643313 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b184df-5df4-4f07-b204-60da71a85731 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:18:57.676954 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb95197-4537-4c3b-a923-b136e1a856e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:18:57.677907 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb95197-4537-4c3b-a923-b136e1a856e6 None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:57.678389 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb95197-4537-4c3b-a923-b136e1a856e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:57.678759 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb95197-4537-4c3b-a923-b136e1a856e6 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:57.681520 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90574f9d-f451-441c-93e3-293e6e3d0741 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:18:57.682336 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90574f9d-f451-441c-93e3-293e6e3d0741 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:57.683313 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90574f9d-f451-441c-93e3-293e6e3d0741 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:57.683757 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90574f9d-f451-441c-93e3-293e6e3d0741 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:57.688590 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb95197-4537-4c3b-a923-b136e1a856e6 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:18:57.690185 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb95197-4537-4c3b-a923-b136e1a856e6 None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:18:57.705411 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application [None req-6eb95197-4537-4c3b-a923-b136e1a856e6 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jun 15 10:18:57.705411 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:18:57.705411 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:18:57.705411 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:18:57.705411 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:18:57.705411 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:18:57.705411 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:18:57.705411 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:18:57.705411 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:18:57.705411 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:18:57.705411 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:18:57.705411 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:18:57.705411 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 15 10:18:57.705411 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 15 10:18:57.705411 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 15 10:18:57.705411 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 15 10:18:57.706647 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:18:57.706647 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:18:57.706647 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 15 10:18:57.706647 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 10:18:57.706647 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 15 10:18:57.706647 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 10:18:57.706647 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:18:57.706647 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:18:57.706647 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:18:57.706647 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:18:57.706647 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:18:57.706647 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application with Lock( Jun 15 10:18:57.706647 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:18:57.706647 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:18:57.706647 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:18:57.706647 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:18:57.706647 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:18:57.706647 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:18:57.707886 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:18:57.707886 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:18:57.707886 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 15 10:18:57.707886 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 15 10:18:57.707886 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 15 10:18:57.707886 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 15 10:18:57.707886 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 15 10:18:57.707886 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 15 10:18:57.707886 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jun 15 10:18:57.707886 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application Jun 15 10:18:57.713671 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 329/658] 10.222.0.49 () {56 vars in 1102 bytes} [Wed Jun 15 10:18:57 2022] GET /identity/v3/users/neutron => generated 85 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:18:57.713671 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb95197-4537-4c3b-a923-b136e1a856e6 None admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:18:57.766776 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90574f9d-f451-441c-93e3-293e6e3d0741 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:18:57.768557 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90574f9d-f451-441c-93e3-293e6e3d0741 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:18:57.811958 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=100 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:18:57.830403 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb95197-4537-4c3b-a923-b136e1a856e6 None admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:18:57.864838 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80bab8d-3be3-4af7-b751-fda65a341a32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:18:57.865635 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80bab8d-3be3-4af7-b751-fda65a341a32 None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:57.865923 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80bab8d-3be3-4af7-b751-fda65a341a32 None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:57.866198 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80bab8d-3be3-4af7-b751-fda65a341a32 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:57.877574 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c80bab8d-3be3-4af7-b751-fda65a341a32 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:18:57.879023 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c80bab8d-3be3-4af7-b751-fda65a341a32 None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:18:57.886589 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:18:57.895123 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 330/659] 10.222.0.49 () {58 vars in 1282 bytes} [Wed Jun 15 10:18:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 15 10:18:57.901480 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90574f9d-f451-441c-93e3-293e6e3d0741 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:18:57.901638 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application [None req-c80bab8d-3be3-4af7-b751-fda65a341a32 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 10:18:57.901638 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:18:57.901638 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:18:57.901638 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:18:57.901638 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:18:57.901638 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:18:57.901638 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:18:57.901638 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:18:57.901638 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:18:57.901638 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:18:57.901638 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:18:57.901638 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:18:57.901638 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 10:18:57.901638 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 10:18:57.901638 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 10:18:57.902758 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 10:18:57.902758 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:18:57.902758 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:18:57.902758 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:18:57.902758 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:18:57.902758 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:18:57.902758 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:18:57.902758 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:18:57.902758 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application with Lock( Jun 15 10:18:57.902758 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:18:57.902758 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:18:57.902758 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:18:57.902758 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:18:57.902758 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:18:57.902758 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:18:57.902758 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:18:57.902758 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:18:57.903876 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 10:18:57.903876 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 10:18:57.903876 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 10:18:57.903876 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application Jun 15 10:18:57.903876 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 330/660] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Jun 15 10:18:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:18:57.903876 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9065412f-f5d6-4a53-b442-dd003898d122 None None] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:57.903876 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9065412f-f5d6-4a53-b442-dd003898d122 None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:57.903876 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9065412f-f5d6-4a53-b442-dd003898d122 None None] PATH_INFO: `/` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:57.903876 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 331/661] 10.222.0.49 () {54 vars in 926 bytes} [Wed Jun 15 10:18:57 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:18:57.908909 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53590e65-a01d-47d5-800d-c3e3a0f9243d None None] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:57.909198 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53590e65-a01d-47d5-800d-c3e3a0f9243d None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:57.909479 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53590e65-a01d-47d5-800d-c3e3a0f9243d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:57.927874 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.auth.core [None req-53590e65-a01d-47d5-800d-c3e3a0f9243d None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:18:57.929075 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-53590e65-a01d-47d5-800d-c3e3a0f9243d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:18:57.999505 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 332/662] 10.222.0.49 () {58 vars in 1023 bytes} [Wed Jun 15 10:18:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 15 10:18:58.000351 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53590e65-a01d-47d5-800d-c3e3a0f9243d None None] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:18:58.027082 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90574f9d-f451-441c-93e3-293e6e3d0741 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:18:58.070635 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=100 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:18:58.071177 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72b5101-a3bc-484c-b6d4-aa2da92d50d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:18:58.071858 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72b5101-a3bc-484c-b6d4-aa2da92d50d4 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:58.072141 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72b5101-a3bc-484c-b6d4-aa2da92d50d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:58.072418 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72b5101-a3bc-484c-b6d4-aa2da92d50d4 None admin] PATH_INFO: `/v3/users` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:58.073413 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72b5101-a3bc-484c-b6d4-aa2da92d50d4 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:18:58.073780 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72b5101-a3bc-484c-b6d4-aa2da92d50d4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:18:58.075223 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72b5101-a3bc-484c-b6d4-aa2da92d50d4 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:18:58.092545 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 331/663] 10.222.0.49 () {56 vars in 1139 bytes} [Wed Jun 15 10:18:57 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:18:58.093364 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72b5101-a3bc-484c-b6d4-aa2da92d50d4 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:18:58.109559 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53590e65-a01d-47d5-800d-c3e3a0f9243d None None] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:18:58.149346 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6e9746-f0ec-4478-a8b0-77aeaf0b0b7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:18:58.149938 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6e9746-f0ec-4478-a8b0-77aeaf0b0b7a None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:58.150224 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6e9746-f0ec-4478-a8b0-77aeaf0b0b7a None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:58.150502 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6e9746-f0ec-4478-a8b0-77aeaf0b0b7a None admin] PATH_INFO: `/v3/domains` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:58.151250 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6e9746-f0ec-4478-a8b0-77aeaf0b0b7a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:18:58.151557 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6e9746-f0ec-4478-a8b0-77aeaf0b0b7a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:18:58.151950 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:18:58.152926 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6e9746-f0ec-4478-a8b0-77aeaf0b0b7a None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:18:58.165025 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 333/664] 10.222.0.49 () {56 vars in 1109 bytes} [Wed Jun 15 10:18:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:18:58.166020 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6e9746-f0ec-4478-a8b0-77aeaf0b0b7a None admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:18:58.203882 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72b5101-a3bc-484c-b6d4-aa2da92d50d4 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:18:58.240940 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5b52f7-beed-4e9e-bf45-56c3bb4916df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:18:58.241731 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5b52f7-beed-4e9e-bf45-56c3bb4916df None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:58.242032 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5b52f7-beed-4e9e-bf45-56c3bb4916df None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:58.242275 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5b52f7-beed-4e9e-bf45-56c3bb4916df None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:58.253183 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b5b52f7-beed-4e9e-bf45-56c3bb4916df None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:18:58.254627 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b5b52f7-beed-4e9e-bf45-56c3bb4916df None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:18:58.271834 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application [None req-1b5b52f7-beed-4e9e-bf45-56c3bb4916df None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 10:18:58.271834 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:18:58.271834 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:18:58.271834 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:18:58.271834 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:18:58.271834 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:18:58.271834 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:18:58.271834 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:18:58.271834 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:18:58.271834 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:18:58.271834 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:18:58.271834 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:18:58.271834 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 10:18:58.271834 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 10:18:58.271834 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 10:18:58.272755 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 10:18:58.272755 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:18:58.272755 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:18:58.272755 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:18:58.272755 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:18:58.272755 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:18:58.272755 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:18:58.272755 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:18:58.272755 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application with Lock( Jun 15 10:18:58.272755 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:18:58.272755 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:18:58.272755 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:18:58.272755 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:18:58.272755 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:18:58.272755 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:18:58.272755 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:18:58.272755 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:18:58.273683 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 10:18:58.273683 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 10:18:58.273683 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 10:18:58.273683 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Jun 15 10:18:58.273683 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 332/665] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Jun 15 10:18:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:18:58.273961 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5b52f7-beed-4e9e-bf45-56c3bb4916df None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:18:58.283877 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6e9746-f0ec-4478-a8b0-77aeaf0b0b7a None admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:18:58.312463 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=100 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:18:58.321240 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79631ef6-9d8c-48bb-b5fd-b8a8bd28203a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:18:58.322031 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79631ef6-9d8c-48bb-b5fd-b8a8bd28203a None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:58.322315 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79631ef6-9d8c-48bb-b5fd-b8a8bd28203a None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:58.322595 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79631ef6-9d8c-48bb-b5fd-b8a8bd28203a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:58.333421 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79631ef6-9d8c-48bb-b5fd-b8a8bd28203a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 10:18:58.341698 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application [None req-79631ef6-9d8c-48bb-b5fd-b8a8bd28203a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 10:18:58.341698 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:18:58.341698 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:18:58.341698 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:18:58.341698 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:18:58.341698 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:18:58.341698 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:18:58.341698 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:18:58.341698 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:18:58.341698 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:18:58.341698 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:18:58.341698 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:18:58.341698 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 10:18:58.341698 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 10:18:58.341698 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 10:18:58.341698 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 10:18:58.342645 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 10:18:58.342645 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 10:18:58.342645 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 10:18:58.342645 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 10:18:58.342645 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:18:58.342645 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:18:58.342645 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:18:58.342645 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:18:58.342645 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:18:58.342645 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:18:58.342645 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:18:58.342645 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application with Lock( Jun 15 10:18:58.342645 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:18:58.342645 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:18:58.342645 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:18:58.342645 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:18:58.343513 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:18:58.343513 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:18:58.343513 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:18:58.343513 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:18:58.343513 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 10:18:58.343513 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 10:18:58.343513 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 10:18:58.343513 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 10:18:58.343513 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 10:18:58.343513 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 10:18:58.343513 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 10:18:58.343513 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application Jun 15 10:18:58.343513 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 334/666] 10.222.0.49 () {56 vars in 1096 bytes} [Wed Jun 15 10:18:58 2022] GET /identity/v3/roles/admin => generated 83 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:18:58.344223 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79631ef6-9d8c-48bb-b5fd-b8a8bd28203a None admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:18:58.388131 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5b52f7-beed-4e9e-bf45-56c3bb4916df None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:18:58.396919 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:18:58.425321 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89ce3ce-0e7a-4e38-9d52-426ce5f6fce8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:18:58.426457 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89ce3ce-0e7a-4e38-9d52-426ce5f6fce8 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:58.426905 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89ce3ce-0e7a-4e38-9d52-426ce5f6fce8 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:58.427364 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89ce3ce-0e7a-4e38-9d52-426ce5f6fce8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:58.455961 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79631ef6-9d8c-48bb-b5fd-b8a8bd28203a None admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:18:58.489712 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60bb9e56-aa97-4736-aa92-a3920f817244 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:18:58.490503 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bb9e56-aa97-4736-aa92-a3920f817244 None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:58.490785 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bb9e56-aa97-4736-aa92-a3920f817244 None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:58.491059 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bb9e56-aa97-4736-aa92-a3920f817244 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:58.496892 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89ce3ce-0e7a-4e38-9d52-426ce5f6fce8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:18:58.498184 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60bb9e56-aa97-4736-aa92-a3920f817244 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:18:58.498655 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89ce3ce-0e7a-4e38-9d52-426ce5f6fce8 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:18:58.499665 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60bb9e56-aa97-4736-aa92-a3920f817244 None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:18:58.516474 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application [None req-60bb9e56-aa97-4736-aa92-a3920f817244 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 10:18:58.516474 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:18:58.516474 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:18:58.516474 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:18:58.516474 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:18:58.516474 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:18:58.516474 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:18:58.516474 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:18:58.516474 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:18:58.516474 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:18:58.516474 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:18:58.516474 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:18:58.516474 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 15 10:18:58.516474 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 15 10:18:58.516474 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 15 10:18:58.517563 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 15 10:18:58.517563 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:18:58.517563 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:18:58.517563 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:18:58.517563 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:18:58.517563 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:18:58.517563 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:18:58.517563 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:18:58.517563 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application with Lock( Jun 15 10:18:58.517563 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:18:58.517563 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:18:58.517563 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:18:58.517563 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:18:58.517563 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:18:58.517563 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:18:58.517563 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:18:58.517563 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:18:58.523072 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 15 10:18:58.523072 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 15 10:18:58.523072 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 15 10:18:58.523072 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 15 10:18:58.523072 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 15 10:18:58.523072 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 15 10:18:58.523072 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 10:18:58.523072 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application Jun 15 10:18:58.523072 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 335/667] 10.222.0.49 () {56 vars in 1111 bytes} [Wed Jun 15 10:18:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:18:58.523072 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60bb9e56-aa97-4736-aa92-a3920f817244 None admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:18:58.568352 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=100 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:18:58.607694 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 333/668] 10.222.0.49 () {58 vars in 1282 bytes} [Wed Jun 15 10:18:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 15 10:18:58.608756 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89ce3ce-0e7a-4e38-9d52-426ce5f6fce8 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:18:58.620697 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:18:58.624866 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60bb9e56-aa97-4736-aa92-a3920f817244 None admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:18:58.659043 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f996135-d6de-46cf-905f-00241af68e49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:18:58.659735 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f996135-d6de-46cf-905f-00241af68e49 None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:58.660015 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f996135-d6de-46cf-905f-00241af68e49 None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:58.660285 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f996135-d6de-46cf-905f-00241af68e49 None admin] PATH_INFO: `/v3/domains` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:58.660994 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f996135-d6de-46cf-905f-00241af68e49 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:18:58.661296 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f996135-d6de-46cf-905f-00241af68e49 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:18:58.662672 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f996135-d6de-46cf-905f-00241af68e49 None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:18:58.675121 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 336/669] 10.222.0.49 () {56 vars in 1109 bytes} [Wed Jun 15 10:18:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:18:58.675814 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f996135-d6de-46cf-905f-00241af68e49 None admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:18:58.720009 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89ce3ce-0e7a-4e38-9d52-426ce5f6fce8 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:18:58.761101 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772ee7f8-3cb0-4863-b68f-57507d60ef64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:18:58.761771 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772ee7f8-3cb0-4863-b68f-57507d60ef64 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:58.762051 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772ee7f8-3cb0-4863-b68f-57507d60ef64 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:58.762325 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772ee7f8-3cb0-4863-b68f-57507d60ef64 None admin] PATH_INFO: `/v3/roles` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:58.763499 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-772ee7f8-3cb0-4863-b68f-57507d60ef64 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:18:58.763809 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-772ee7f8-3cb0-4863-b68f-57507d60ef64 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:18:58.765166 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-772ee7f8-3cb0-4863-b68f-57507d60ef64 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:18:58.777599 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 334/670] 10.222.0.49 () {56 vars in 1099 bytes} [Wed Jun 15 10:18:58 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:18:58.777599 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772ee7f8-3cb0-4863-b68f-57507d60ef64 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:18:58.784411 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f996135-d6de-46cf-905f-00241af68e49 None admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:18:58.795642 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=100 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:18:58.822629 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cff3f52-85a8-49a6-9ac7-fca5d86572ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:18:58.823604 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cff3f52-85a8-49a6-9ac7-fca5d86572ac None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:58.823891 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cff3f52-85a8-49a6-9ac7-fca5d86572ac None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:58.824168 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cff3f52-85a8-49a6-9ac7-fca5d86572ac None admin] PATH_INFO: `/v3/projects` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:58.824948 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cff3f52-85a8-49a6-9ac7-fca5d86572ac None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:18:58.825256 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cff3f52-85a8-49a6-9ac7-fca5d86572ac None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:18:58.826688 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cff3f52-85a8-49a6-9ac7-fca5d86572ac None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:18:58.843435 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 337/671] 10.222.0.49 () {56 vars in 1148 bytes} [Wed Jun 15 10:18:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:18:58.843435 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cff3f52-85a8-49a6-9ac7-fca5d86572ac None admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:18:58.871000 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:18:58.909543 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772ee7f8-3cb0-4863-b68f-57507d60ef64 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:18:58.946013 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703461f8-7e05-4c4b-9a0e-0581fae275b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:18:58.946486 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cff3f52-85a8-49a6-9ac7-fca5d86572ac None admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:18:58.946740 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703461f8-7e05-4c4b-9a0e-0581fae275b2 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:58.947029 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703461f8-7e05-4c4b-9a0e-0581fae275b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:58.947337 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703461f8-7e05-4c4b-9a0e-0581fae275b2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:58.958294 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-703461f8-7e05-4c4b-9a0e-0581fae275b2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:18:58.959780 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-703461f8-7e05-4c4b-9a0e-0581fae275b2 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:18:58.977177 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application [None req-703461f8-7e05-4c4b-9a0e-0581fae275b2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 10:18:58.977177 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:18:58.977177 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:18:58.977177 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:18:58.977177 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:18:58.977177 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:18:58.977177 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:18:58.977177 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:18:58.977177 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:18:58.977177 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:18:58.977177 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:18:58.977177 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:18:58.977177 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 10:18:58.977177 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 10:18:58.977177 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 10:18:58.978097 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 10:18:58.978097 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:18:58.978097 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:18:58.978097 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:18:58.978097 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:18:58.978097 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:18:58.978097 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:18:58.978097 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:18:58.978097 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application with Lock( Jun 15 10:18:58.978097 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:18:58.978097 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:18:58.978097 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:18:58.978097 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:18:58.978097 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:18:58.978097 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:18:58.978097 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:18:58.978097 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:18:58.978986 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 10:18:58.978986 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 10:18:58.978986 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 10:18:58.978986 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Jun 15 10:18:58.978986 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 335/672] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Jun 15 10:18:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:18:58.984142 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703461f8-7e05-4c4b-9a0e-0581fae275b2 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:18:58.985469 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5089c2-3aa0-480c-b915-abb308893b3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:18:58.986276 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5089c2-3aa0-480c-b915-abb308893b3b None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:58.986561 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5089c2-3aa0-480c-b915-abb308893b3b None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:58.986837 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5089c2-3aa0-480c-b915-abb308893b3b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:58.994566 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c5089c2-3aa0-480c-b915-abb308893b3b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:18:58.996824 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c5089c2-3aa0-480c-b915-abb308893b3b None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:18:59.010644 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application [None req-0c5089c2-3aa0-480c-b915-abb308893b3b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 10:18:59.010644 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:18:59.010644 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:18:59.010644 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:18:59.010644 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:18:59.010644 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:18:59.010644 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:18:59.010644 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:18:59.010644 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:18:59.010644 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:18:59.010644 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:18:59.010644 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:18:59.010644 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 15 10:18:59.010644 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 15 10:18:59.010644 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 15 10:18:59.011766 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 15 10:18:59.011766 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:18:59.011766 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:18:59.011766 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:18:59.011766 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:18:59.011766 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:18:59.011766 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:18:59.011766 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:18:59.011766 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application with Lock( Jun 15 10:18:59.011766 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:18:59.011766 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:18:59.011766 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:18:59.011766 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:18:59.011766 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:18:59.011766 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:18:59.011766 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:18:59.011766 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:18:59.012683 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 15 10:18:59.012683 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 15 10:18:59.012683 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 15 10:18:59.012683 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 15 10:18:59.012683 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 15 10:18:59.012683 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 15 10:18:59.012683 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 10:18:59.012683 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application Jun 15 10:18:59.012683 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 338/673] 10.222.0.49 () {56 vars in 1111 bytes} [Wed Jun 15 10:18:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:18:59.012683 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5089c2-3aa0-480c-b915-abb308893b3b None admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:18:59.047547 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=100 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:18:59.099258 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703461f8-7e05-4c4b-9a0e-0581fae275b2 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:18:59.120694 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:18:59.124657 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5089c2-3aa0-480c-b915-abb308893b3b None admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:18:59.135442 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613fa198-7f6e-4b8d-afcf-baf2b48d11d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:18:59.136274 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613fa198-7f6e-4b8d-afcf-baf2b48d11d1 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:59.136565 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613fa198-7f6e-4b8d-afcf-baf2b48d11d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:59.136849 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613fa198-7f6e-4b8d-afcf-baf2b48d11d1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:59.148520 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-613fa198-7f6e-4b8d-afcf-baf2b48d11d1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:18:59.149978 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-613fa198-7f6e-4b8d-afcf-baf2b48d11d1 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:18:59.161108 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c41a765-db81-4190-a309-cd144d4e42d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:18:59.161769 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c41a765-db81-4190-a309-cd144d4e42d4 None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:59.162052 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c41a765-db81-4190-a309-cd144d4e42d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:59.162328 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c41a765-db81-4190-a309-cd144d4e42d4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:59.163183 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c41a765-db81-4190-a309-cd144d4e42d4 None admin] RBAC: Adding query filter params (role.id=16e76452bb5d4ce68a119ad3c9bde41c, scope.project.id=559b3e713f5844b6ae37151d56dd419f, user.id=91f2fe3be62e49548eae208e57bf2a93) {{(pid=80119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:18:59.163498 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c41a765-db81-4190-a309-cd144d4e42d4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:18:59.164923 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c41a765-db81-4190-a309-cd144d4e42d4 None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:18:59.168485 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application [None req-613fa198-7f6e-4b8d-afcf-baf2b48d11d1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 10:18:59.168485 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:18:59.168485 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:18:59.168485 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:18:59.168485 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:18:59.168485 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:18:59.168485 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:18:59.168485 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:18:59.168485 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:18:59.168485 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:18:59.168485 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:18:59.168485 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:18:59.168485 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 10:18:59.168485 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 10:18:59.168485 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 10:18:59.169358 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 10:18:59.169358 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:18:59.169358 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:18:59.169358 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:18:59.169358 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:18:59.169358 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:18:59.169358 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:18:59.169358 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:18:59.169358 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application with Lock( Jun 15 10:18:59.169358 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:18:59.169358 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:18:59.169358 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:18:59.169358 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:18:59.169358 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:18:59.169358 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:18:59.169358 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:18:59.169358 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:18:59.170260 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 10:18:59.170260 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 10:18:59.170260 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 10:18:59.170260 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Jun 15 10:18:59.170260 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 336/674] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Jun 15 10:18:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:18:59.170533 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613fa198-7f6e-4b8d-afcf-baf2b48d11d1 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:18:59.176839 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 339/675] 10.222.0.49 () {56 vars in 1374 bytes} [Wed Jun 15 10:18:59 2022] GET /identity/v3/role_assignments?user.id=91f2fe3be62e49548eae208e57bf2a93&scope.project.id=559b3e713f5844b6ae37151d56dd419f&role.id=16e76452bb5d4ce68a119ad3c9bde41c => generated 608 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:18:59.177954 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c41a765-db81-4190-a309-cd144d4e42d4 None admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:18:59.307700 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613fa198-7f6e-4b8d-afcf-baf2b48d11d1 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:18:59.311725 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=100 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:18:59.321223 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c41a765-db81-4190-a309-cd144d4e42d4 None admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:18:59.349189 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaec0978-24f2-4b3d-9a49-fad7518116e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:18:59.349880 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaec0978-24f2-4b3d-9a49-fad7518116e8 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:59.350163 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaec0978-24f2-4b3d-9a49-fad7518116e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:59.350440 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaec0978-24f2-4b3d-9a49-fad7518116e8 None admin] PATH_INFO: `/v3/domains` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:59.351223 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaec0978-24f2-4b3d-9a49-fad7518116e8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:18:59.351532 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaec0978-24f2-4b3d-9a49-fad7518116e8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:18:59.352924 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaec0978-24f2-4b3d-9a49-fad7518116e8 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:18:59.363290 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d00bd9-75c4-41af-b70d-421a8cd86183 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:18:59.364264 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d00bd9-75c4-41af-b70d-421a8cd86183 None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:59.364553 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d00bd9-75c4-41af-b70d-421a8cd86183 None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:59.364837 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d00bd9-75c4-41af-b70d-421a8cd86183 None admin] PATH_INFO: `/v3/projects` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:59.365629 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d00bd9-75c4-41af-b70d-421a8cd86183 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:18:59.365944 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d00bd9-75c4-41af-b70d-421a8cd86183 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:18:59.367407 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d00bd9-75c4-41af-b70d-421a8cd86183 None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:18:59.367589 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 337/676] 10.222.0.49 () {56 vars in 1109 bytes} [Wed Jun 15 10:18:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:18:59.368558 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaec0978-24f2-4b3d-9a49-fad7518116e8 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:18:59.381594 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 340/677] 10.222.0.49 () {56 vars in 1148 bytes} [Wed Jun 15 10:18:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:18:59.382924 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f51532-6e69-4abe-b224-c6ddd235ba1a None None] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:59.383284 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f51532-6e69-4abe-b224-c6ddd235ba1a None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:59.383567 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f51532-6e69-4abe-b224-c6ddd235ba1a None None] PATH_INFO: `/` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:59.385266 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 341/678] 10.222.0.49 () {54 vars in 926 bytes} [Wed Jun 15 10:18:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:18:59.391145 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f51532-6e69-4abe-b224-c6ddd235ba1a None None] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:18:59.405611 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:18:59.481536 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaec0978-24f2-4b3d-9a49-fad7518116e8 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:18:59.500847 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f51532-6e69-4abe-b224-c6ddd235ba1a None None] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:18:59.517423 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc40661-396a-4aa8-bc80-54904893aa15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:18:59.518265 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc40661-396a-4aa8-bc80-54904893aa15 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:59.518620 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc40661-396a-4aa8-bc80-54904893aa15 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:59.519008 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc40661-396a-4aa8-bc80-54904893aa15 None admin] PATH_INFO: `/v3/domains` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:59.519763 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc40661-396a-4aa8-bc80-54904893aa15 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:18:59.520069 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc40661-396a-4aa8-bc80-54904893aa15 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:18:59.521434 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc40661-396a-4aa8-bc80-54904893aa15 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:18:59.534327 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 338/679] 10.222.0.49 () {56 vars in 1109 bytes} [Wed Jun 15 10:18:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:18:59.534961 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc40661-396a-4aa8-bc80-54904893aa15 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:18:59.540602 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ccfac9-20b3-4f22-9d07-dcf49b988e1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:18:59.541422 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ccfac9-20b3-4f22-9d07-dcf49b988e1a None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:59.541706 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ccfac9-20b3-4f22-9d07-dcf49b988e1a None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:59.542031 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ccfac9-20b3-4f22-9d07-dcf49b988e1a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:59.549109 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ccfac9-20b3-4f22-9d07-dcf49b988e1a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:18:59.551126 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ccfac9-20b3-4f22-9d07-dcf49b988e1a None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:18:59.564790 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application [None req-94ccfac9-20b3-4f22-9d07-dcf49b988e1a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 10:18:59.564790 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:18:59.564790 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:18:59.564790 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:18:59.564790 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:18:59.564790 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:18:59.564790 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:18:59.564790 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:18:59.564790 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:18:59.564790 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:18:59.564790 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:18:59.564790 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:18:59.564790 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 15 10:18:59.564790 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 15 10:18:59.564790 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 15 10:18:59.565695 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 15 10:18:59.565695 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:18:59.565695 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:18:59.565695 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:18:59.565695 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:18:59.565695 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:18:59.565695 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:18:59.565695 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:18:59.565695 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application with Lock( Jun 15 10:18:59.565695 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:18:59.565695 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:18:59.565695 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:18:59.565695 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:18:59.565695 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:18:59.565695 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:18:59.565695 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:18:59.565695 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:18:59.566619 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 15 10:18:59.566619 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 15 10:18:59.566619 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 15 10:18:59.566619 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 15 10:18:59.566619 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 15 10:18:59.566619 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 15 10:18:59.566619 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 10:18:59.566619 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application Jun 15 10:18:59.566619 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 342/680] 10.222.0.49 () {56 vars in 1111 bytes} [Wed Jun 15 10:18:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:18:59.567745 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecd4150-36e0-4d87-b706-d5dc3e1de593 None None] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:59.567745 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecd4150-36e0-4d87-b706-d5dc3e1de593 None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:59.567745 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecd4150-36e0-4d87-b706-d5dc3e1de593 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:59.584624 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.auth.core [None req-aecd4150-36e0-4d87-b706-d5dc3e1de593 None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:18:59.585816 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-aecd4150-36e0-4d87-b706-d5dc3e1de593 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:18:59.594801 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=100 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:18:59.637785 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:18:59.648382 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc40661-396a-4aa8-bc80-54904893aa15 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:18:59.658200 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 343/681] 10.222.0.49 () {58 vars in 1023 bytes} [Wed Jun 15 10:18:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 10:18:59.659342 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aecd4150-36e0-4d87-b706-d5dc3e1de593 None None] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:18:59.695529 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d1c4f6-1b27-4e46-acd8-962aba90599e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:18:59.696437 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d1c4f6-1b27-4e46-acd8-962aba90599e None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:59.696895 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d1c4f6-1b27-4e46-acd8-962aba90599e None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:59.697331 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d1c4f6-1b27-4e46-acd8-962aba90599e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:59.698808 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d1c4f6-1b27-4e46-acd8-962aba90599e None admin] RBAC: Adding query filter params (role.id=16e76452bb5d4ce68a119ad3c9bde41c, scope.project.id=559b3e713f5844b6ae37151d56dd419f, user.id=20d1327553404b80aecee87c78fcb52b) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:18:59.699302 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d1c4f6-1b27-4e46-acd8-962aba90599e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:18:59.700945 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d1c4f6-1b27-4e46-acd8-962aba90599e None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:18:59.707260 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 339/682] 10.222.0.49 () {56 vars in 1374 bytes} [Wed Jun 15 10:18:59 2022] GET /identity/v3/role_assignments?user.id=20d1327553404b80aecee87c78fcb52b&scope.project.id=559b3e713f5844b6ae37151d56dd419f&role.id=16e76452bb5d4ce68a119ad3c9bde41c => generated 608 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:18:59.708238 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d1c4f6-1b27-4e46-acd8-962aba90599e None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:18:59.794925 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aecd4150-36e0-4d87-b706-d5dc3e1de593 None None] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:18:59.836110 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9aaeabd-883f-4ee9-97ce-36c28237fbfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:18:59.837486 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9aaeabd-883f-4ee9-97ce-36c28237fbfb None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:59.837875 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9aaeabd-883f-4ee9-97ce-36c28237fbfb None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:59.838251 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9aaeabd-883f-4ee9-97ce-36c28237fbfb None admin] PATH_INFO: `/v3/users/nova` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:59.844791 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d1c4f6-1b27-4e46-acd8-962aba90599e None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:18:59.847011 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9aaeabd-883f-4ee9-97ce-36c28237fbfb None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:18:59.848732 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9aaeabd-883f-4ee9-97ce-36c28237fbfb None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:18:59.865324 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application [None req-d9aaeabd-883f-4ee9-97ce-36c28237fbfb None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 15 10:18:59.865324 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:18:59.865324 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:18:59.865324 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:18:59.865324 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:18:59.865324 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:18:59.865324 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:18:59.865324 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:18:59.865324 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:18:59.865324 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:18:59.865324 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:18:59.865324 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:18:59.865324 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 15 10:18:59.865324 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 15 10:18:59.865324 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 15 10:18:59.865324 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 15 10:18:59.866544 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:18:59.866544 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:18:59.866544 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 15 10:18:59.866544 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 10:18:59.866544 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 15 10:18:59.866544 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 10:18:59.866544 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:18:59.866544 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:18:59.866544 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:18:59.866544 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:18:59.866544 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:18:59.866544 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application with Lock( Jun 15 10:18:59.866544 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:18:59.866544 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:18:59.866544 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:18:59.866544 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:18:59.866544 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:18:59.866544 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:18:59.867796 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:18:59.867796 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:18:59.867796 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 15 10:18:59.867796 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 15 10:18:59.867796 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 15 10:18:59.867796 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 15 10:18:59.867796 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 15 10:18:59.867796 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 15 10:18:59.867796 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 15 10:18:59.867796 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application Jun 15 10:18:59.869658 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 344/683] 10.222.0.49 () {56 vars in 1093 bytes} [Wed Jun 15 10:18:59 2022] GET /identity/v3/users/nova => generated 82 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:18:59.871135 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9aaeabd-883f-4ee9-97ce-36c28237fbfb None admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:18:59.874368 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=100 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:18:59.901043 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f970d2-6485-4013-adbe-a43faf3ad330 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:18:59.901936 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f970d2-6485-4013-adbe-a43faf3ad330 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:18:59.902329 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f970d2-6485-4013-adbe-a43faf3ad330 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:18:59.902703 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f970d2-6485-4013-adbe-a43faf3ad330 None admin] PATH_INFO: `/v3/projects` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:18:59.903542 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f970d2-6485-4013-adbe-a43faf3ad330 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:18:59.903852 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f970d2-6485-4013-adbe-a43faf3ad330 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:18:59.905295 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f970d2-6485-4013-adbe-a43faf3ad330 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:18:59.918375 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 340/684] 10.222.0.49 () {56 vars in 1148 bytes} [Wed Jun 15 10:18:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:18:59.919107 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f970d2-6485-4013-adbe-a43faf3ad330 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:18:59.939238 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:19:00.006652 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9aaeabd-883f-4ee9-97ce-36c28237fbfb None admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:19:00.041465 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f970d2-6485-4013-adbe-a43faf3ad330 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:19:00.050320 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3727ff4-4a19-4a3e-ab1f-f4980a117035 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:19:00.051410 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3727ff4-4a19-4a3e-ab1f-f4980a117035 None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:00.051694 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3727ff4-4a19-4a3e-ab1f-f4980a117035 None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:00.052068 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3727ff4-4a19-4a3e-ab1f-f4980a117035 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:00.064315 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3727ff4-4a19-4a3e-ab1f-f4980a117035 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 10:19:00.072559 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application [None req-e3727ff4-4a19-4a3e-ab1f-f4980a117035 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 15 10:19:00.072559 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:19:00.072559 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:19:00.072559 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:19:00.072559 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:19:00.072559 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:19:00.072559 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:19:00.072559 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:19:00.072559 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:19:00.072559 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:19:00.072559 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:19:00.072559 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:19:00.072559 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 10:19:00.072559 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 10:19:00.072559 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 10:19:00.072559 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 10:19:00.074132 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 10:19:00.074132 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 10:19:00.074132 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 10:19:00.074132 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 10:19:00.074132 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:19:00.074132 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:19:00.074132 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:19:00.074132 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:19:00.074132 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:19:00.074132 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:19:00.074132 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:19:00.074132 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application with Lock( Jun 15 10:19:00.074132 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:19:00.074132 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:19:00.074132 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:19:00.074132 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:19:00.075525 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:19:00.075525 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:19:00.075525 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:19:00.075525 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:19:00.075525 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 10:19:00.075525 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 10:19:00.075525 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 10:19:00.075525 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 10:19:00.075525 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 10:19:00.075525 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 10:19:00.075525 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 15 10:19:00.075525 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application Jun 15 10:19:00.075525 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 345/685] 10.222.0.49 () {56 vars in 1102 bytes} [Wed Jun 15 10:18:59 2022] GET /identity/v3/roles/service => generated 85 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:19:00.076242 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3727ff4-4a19-4a3e-ab1f-f4980a117035 None admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:19:00.085358 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de648b16-79d9-4d3d-8de5-ccbe4203a692 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:19:00.086317 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de648b16-79d9-4d3d-8de5-ccbe4203a692 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:00.086735 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de648b16-79d9-4d3d-8de5-ccbe4203a692 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:00.087264 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de648b16-79d9-4d3d-8de5-ccbe4203a692 None admin] PATH_INFO: `/v3/users` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:00.088272 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de648b16-79d9-4d3d-8de5-ccbe4203a692 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:19:00.088926 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de648b16-79d9-4d3d-8de5-ccbe4203a692 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:19:00.090511 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de648b16-79d9-4d3d-8de5-ccbe4203a692 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:19:00.115383 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 341/686] 10.222.0.49 () {56 vars in 1133 bytes} [Wed Jun 15 10:18:59 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:19:00.116041 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de648b16-79d9-4d3d-8de5-ccbe4203a692 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:19:00.146774 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=100 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:19:00.191584 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3727ff4-4a19-4a3e-ab1f-f4980a117035 None admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:19:00.212735 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:19:00.226949 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6216b204-d8de-4968-9cf6-46cfb37cf10a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:19:00.227657 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6216b204-d8de-4968-9cf6-46cfb37cf10a None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:00.227946 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6216b204-d8de-4968-9cf6-46cfb37cf10a None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:00.228229 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6216b204-d8de-4968-9cf6-46cfb37cf10a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:00.229055 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6216b204-d8de-4968-9cf6-46cfb37cf10a None admin] RBAC: Adding query filter params (role.id=16e76452bb5d4ce68a119ad3c9bde41c, scope.project.id=559b3e713f5844b6ae37151d56dd419f, user.id=0cb44a17d205431793e9a8791e477a32) {{(pid=80119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:19:00.229899 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6216b204-d8de-4968-9cf6-46cfb37cf10a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:19:00.231354 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6216b204-d8de-4968-9cf6-46cfb37cf10a None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:19:00.237765 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 346/687] 10.222.0.49 () {56 vars in 1374 bytes} [Wed Jun 15 10:19:00 2022] GET /identity/v3/role_assignments?user.id=0cb44a17d205431793e9a8791e477a32&scope.project.id=559b3e713f5844b6ae37151d56dd419f&role.id=16e76452bb5d4ce68a119ad3c9bde41c => generated 608 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:19:00.238659 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6216b204-d8de-4968-9cf6-46cfb37cf10a None admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:19:00.247010 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de648b16-79d9-4d3d-8de5-ccbe4203a692 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:19:00.321478 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9190d2d-e96f-41f2-be26-7954e5266687 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:19:00.322245 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9190d2d-e96f-41f2-be26-7954e5266687 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:00.322528 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9190d2d-e96f-41f2-be26-7954e5266687 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:00.322801 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9190d2d-e96f-41f2-be26-7954e5266687 None admin] PATH_INFO: `/v3/roles` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:00.323709 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9190d2d-e96f-41f2-be26-7954e5266687 None admin] RBAC: Adding query filter params (name=service) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:19:00.324019 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9190d2d-e96f-41f2-be26-7954e5266687 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:19:00.326352 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9190d2d-e96f-41f2-be26-7954e5266687 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:19:00.349755 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 342/688] 10.222.0.49 () {56 vars in 1103 bytes} [Wed Jun 15 10:19:00 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:19:00.352238 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9190d2d-e96f-41f2-be26-7954e5266687 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:19:00.413724 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6216b204-d8de-4968-9cf6-46cfb37cf10a None admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:19:00.443571 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=100 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:19:00.454206 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9604532a-eddc-4ce8-b3ca-ab8d74fef408 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:19:00.455217 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9604532a-eddc-4ce8-b3ca-ab8d74fef408 None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:00.455678 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9604532a-eddc-4ce8-b3ca-ab8d74fef408 None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:00.456102 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9604532a-eddc-4ce8-b3ca-ab8d74fef408 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:00.494337 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9190d2d-e96f-41f2-be26-7954e5266687 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:19:00.534371 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9604532a-eddc-4ce8-b3ca-ab8d74fef408 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:19:00.535905 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9604532a-eddc-4ce8-b3ca-ab8d74fef408 None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:19:00.540182 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07406216-6ae9-4654-8c1c-7516254a74f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:19:00.541240 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07406216-6ae9-4654-8c1c-7516254a74f7 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:00.541731 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07406216-6ae9-4654-8c1c-7516254a74f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:00.542776 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07406216-6ae9-4654-8c1c-7516254a74f7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:00.546475 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:19:00.560331 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07406216-6ae9-4654-8c1c-7516254a74f7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:19:00.563037 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07406216-6ae9-4654-8c1c-7516254a74f7 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:19:00.588451 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application [None req-07406216-6ae9-4654-8c1c-7516254a74f7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 10:19:00.588451 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:19:00.588451 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:19:00.588451 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:19:00.588451 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:19:00.588451 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:19:00.588451 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:19:00.588451 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:19:00.588451 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:19:00.588451 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:19:00.588451 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:19:00.588451 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:19:00.588451 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 10:19:00.588451 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 10:19:00.588451 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 10:19:00.589947 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 10:19:00.589947 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:19:00.589947 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:19:00.589947 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:19:00.589947 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:19:00.589947 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:19:00.589947 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:19:00.589947 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:19:00.589947 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application with Lock( Jun 15 10:19:00.589947 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:19:00.589947 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:19:00.589947 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:19:00.589947 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:19:00.589947 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:19:00.589947 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:19:00.589947 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:19:00.589947 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:19:00.590866 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 10:19:00.590866 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 10:19:00.590866 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 10:19:00.590866 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Jun 15 10:19:00.593626 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 343/689] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Jun 15 10:19:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:19:00.599337 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07406216-6ae9-4654-8c1c-7516254a74f7 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:19:00.657102 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 347/690] 10.222.0.49 () {58 vars in 1282 bytes} [Wed Jun 15 10:19:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 419 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 15 10:19:00.663237 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9604532a-eddc-4ce8-b3ca-ab8d74fef408 None admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:19:00.715453 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=100 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:19:00.778352 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07406216-6ae9-4654-8c1c-7516254a74f7 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:19:00.821170 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2516270-bf13-4b9a-8a00-d59f1ce87402 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:19:00.822193 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2516270-bf13-4b9a-8a00-d59f1ce87402 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:00.822712 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2516270-bf13-4b9a-8a00-d59f1ce87402 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:00.823784 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2516270-bf13-4b9a-8a00-d59f1ce87402 None admin] PATH_INFO: `/v3/domains` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:00.824739 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2516270-bf13-4b9a-8a00-d59f1ce87402 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:19:00.825251 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2516270-bf13-4b9a-8a00-d59f1ce87402 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:19:00.826902 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2516270-bf13-4b9a-8a00-d59f1ce87402 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:19:00.840104 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 344/691] 10.222.0.49 () {56 vars in 1109 bytes} [Wed Jun 15 10:19:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 10:19:00.845597 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2516270-bf13-4b9a-8a00-d59f1ce87402 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:19:00.904233 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9604532a-eddc-4ce8-b3ca-ab8d74fef408 None admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:19:00.935492 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:19:00.974322 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fee74f2-690b-4889-8540-1471926a7c16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:19:00.975506 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fee74f2-690b-4889-8540-1471926a7c16 None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:00.976080 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fee74f2-690b-4889-8540-1471926a7c16 None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:00.977341 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fee74f2-690b-4889-8540-1471926a7c16 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:00.996490 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fee74f2-690b-4889-8540-1471926a7c16 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:19:01.000327 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fee74f2-690b-4889-8540-1471926a7c16 None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:19:01.023012 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application [None req-1fee74f2-690b-4889-8540-1471926a7c16 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 10:19:01.023012 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:19:01.023012 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:19:01.023012 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:19:01.023012 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:19:01.023012 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:19:01.023012 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:19:01.023012 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:19:01.023012 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:19:01.023012 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:19:01.023012 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:19:01.023012 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:19:01.023012 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 10:19:01.023012 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 10:19:01.023012 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 10:19:01.024103 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 10:19:01.024103 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:19:01.024103 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:19:01.024103 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:19:01.024103 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:19:01.024103 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:19:01.024103 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:19:01.024103 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:19:01.024103 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application with Lock( Jun 15 10:19:01.024103 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:19:01.024103 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:19:01.024103 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:19:01.024103 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:19:01.024103 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:19:01.024103 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:19:01.024103 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:19:01.024103 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:19:01.025019 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 10:19:01.025019 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 10:19:01.025019 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 10:19:01.025019 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application Jun 15 10:19:01.027787 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 348/692] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Jun 15 10:19:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 365 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:19:01.036084 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fee74f2-690b-4889-8540-1471926a7c16 None admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:19:01.041551 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2516270-bf13-4b9a-8a00-d59f1ce87402 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:19:01.079631 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799f316c-f86e-4dc3-8504-2fcec8015c20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:19:01.080444 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799f316c-f86e-4dc3-8504-2fcec8015c20 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:01.080729 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799f316c-f86e-4dc3-8504-2fcec8015c20 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:01.081007 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799f316c-f86e-4dc3-8504-2fcec8015c20 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:01.088787 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799f316c-f86e-4dc3-8504-2fcec8015c20 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:19:01.090742 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799f316c-f86e-4dc3-8504-2fcec8015c20 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:19:01.112128 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application [None req-799f316c-f86e-4dc3-8504-2fcec8015c20 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 15 10:19:01.112128 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:19:01.112128 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:19:01.112128 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:19:01.112128 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:19:01.112128 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:19:01.112128 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:19:01.112128 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:19:01.112128 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:19:01.112128 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:19:01.112128 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:19:01.112128 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:19:01.112128 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 15 10:19:01.112128 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 15 10:19:01.112128 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 15 10:19:01.112128 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 15 10:19:01.113353 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:19:01.113353 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:19:01.113353 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 15 10:19:01.113353 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 10:19:01.113353 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 15 10:19:01.113353 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 10:19:01.113353 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:19:01.113353 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:19:01.113353 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:19:01.113353 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:19:01.113353 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:19:01.113353 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application with Lock( Jun 15 10:19:01.113353 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:19:01.113353 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:19:01.113353 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:19:01.113353 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:19:01.113353 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:19:01.113353 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:19:01.114399 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:19:01.114399 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:19:01.114399 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 15 10:19:01.114399 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 15 10:19:01.114399 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 15 10:19:01.114399 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 15 10:19:01.114399 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 15 10:19:01.114399 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 15 10:19:01.114399 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 15 10:19:01.114399 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Jun 15 10:19:01.114399 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 345/693] 10.222.0.49 () {56 vars in 1096 bytes} [Wed Jun 15 10:19:00 2022] GET /identity/v3/users/swift => generated 83 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 10:19:01.114399 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799f316c-f86e-4dc3-8504-2fcec8015c20 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:19:01.220974 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799f316c-f86e-4dc3-8504-2fcec8015c20 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:19:01.230638 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:19:01.234603 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=100 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:19:01.258442 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11dd217c-48c5-4040-b8c9-706947b151b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:19:01.259434 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11dd217c-48c5-4040-b8c9-706947b151b2 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:01.259866 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11dd217c-48c5-4040-b8c9-706947b151b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:01.260309 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11dd217c-48c5-4040-b8c9-706947b151b2 None admin] PATH_INFO: `/v3/users` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:01.261851 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11dd217c-48c5-4040-b8c9-706947b151b2 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:19:01.262329 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11dd217c-48c5-4040-b8c9-706947b151b2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:19:01.263949 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11dd217c-48c5-4040-b8c9-706947b151b2 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:19:01.280022 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fee74f2-690b-4889-8540-1471926a7c16 None admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:19:01.288064 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 346/694] 10.222.0.49 () {56 vars in 1135 bytes} [Wed Jun 15 10:19:01 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 10:19:01.293022 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11dd217c-48c5-4040-b8c9-706947b151b2 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:19:01.345547 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6bef28-7381-4cd9-b85b-a8bcc179069b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:19:01.347609 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6bef28-7381-4cd9-b85b-a8bcc179069b None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:01.348991 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6bef28-7381-4cd9-b85b-a8bcc179069b None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:01.349422 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6bef28-7381-4cd9-b85b-a8bcc179069b None admin] PATH_INFO: `/v3/domains` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:01.351700 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb6bef28-7381-4cd9-b85b-a8bcc179069b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:19:01.353344 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb6bef28-7381-4cd9-b85b-a8bcc179069b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:19:01.354933 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb6bef28-7381-4cd9-b85b-a8bcc179069b None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:19:01.382043 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 349/695] 10.222.0.49 () {56 vars in 1109 bytes} [Wed Jun 15 10:19:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 343 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:19:01.393332 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6bef28-7381-4cd9-b85b-a8bcc179069b None admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:19:01.495986 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11dd217c-48c5-4040-b8c9-706947b151b2 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:19:01.568308 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6bef28-7381-4cd9-b85b-a8bcc179069b None admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:19:01.572403 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ecb6472-e403-4ae6-921d-ef5c7da4c7d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:19:01.573128 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecb6472-e403-4ae6-921d-ef5c7da4c7d3 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:01.573911 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecb6472-e403-4ae6-921d-ef5c7da4c7d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:01.574184 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecb6472-e403-4ae6-921d-ef5c7da4c7d3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:01.611165 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:19:01.615517 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88772858-f4df-46a9-8ae5-a188e24f974f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:19:01.616325 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88772858-f4df-46a9-8ae5-a188e24f974f None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:01.616610 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88772858-f4df-46a9-8ae5-a188e24f974f None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:01.616888 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88772858-f4df-46a9-8ae5-a188e24f974f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:01.627423 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88772858-f4df-46a9-8ae5-a188e24f974f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:19:01.629699 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88772858-f4df-46a9-8ae5-a188e24f974f None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:19:01.631829 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=100 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:19:01.651614 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application [None req-88772858-f4df-46a9-8ae5-a188e24f974f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 10:19:01.651614 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:19:01.651614 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:19:01.651614 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:19:01.651614 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:19:01.651614 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:19:01.651614 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:19:01.651614 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:19:01.651614 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:19:01.651614 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:19:01.651614 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:19:01.651614 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:19:01.651614 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 15 10:19:01.651614 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 15 10:19:01.651614 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 15 10:19:01.652607 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 15 10:19:01.652607 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:19:01.652607 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:19:01.652607 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:19:01.652607 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:19:01.652607 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:19:01.652607 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:19:01.652607 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:19:01.652607 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application with Lock( Jun 15 10:19:01.652607 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:19:01.652607 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:19:01.652607 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:19:01.652607 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:19:01.652607 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:19:01.652607 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:19:01.652607 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:19:01.652607 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:19:01.653491 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 15 10:19:01.653491 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 15 10:19:01.653491 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 15 10:19:01.653491 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 15 10:19:01.653491 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 15 10:19:01.653491 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 15 10:19:01.653491 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 10:19:01.653491 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application Jun 15 10:19:01.653491 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 350/696] 10.222.0.49 () {56 vars in 1111 bytes} [Wed Jun 15 10:19:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:19:01.655312 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320a4c55-4632-4d7e-9c02-16a4ea72c5cc None None] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:01.655603 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320a4c55-4632-4d7e-9c02-16a4ea72c5cc None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:01.655884 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320a4c55-4632-4d7e-9c02-16a4ea72c5cc None None] PATH_INFO: `/` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:01.662134 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 351/697] 10.222.0.49 () {54 vars in 926 bytes} [Wed Jun 15 10:19:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:19:01.662134 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320a4c55-4632-4d7e-9c02-16a4ea72c5cc None None] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:19:01.663564 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ecb6472-e403-4ae6-921d-ef5c7da4c7d3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:19:01.665386 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ecb6472-e403-4ae6-921d-ef5c7da4c7d3 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:19:01.782326 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320a4c55-4632-4d7e-9c02-16a4ea72c5cc None None] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:19:01.787606 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 347/698] 10.222.0.49 () {58 vars in 1282 bytes} [Wed Jun 15 10:19:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 495 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 15 10:19:01.789119 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95586fd-af8c-4761-956b-603ca16746d8 None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:01.789627 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95586fd-af8c-4761-956b-603ca16746d8 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:01.790096 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95586fd-af8c-4761-956b-603ca16746d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:01.809486 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-b95586fd-af8c-4761-956b-603ca16746d8 None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:19:01.811109 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-b95586fd-af8c-4761-956b-603ca16746d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:19:01.825820 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7ee6f0-f80c-4e5d-8f17-751f0345a1e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:19:01.826937 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7ee6f0-f80c-4e5d-8f17-751f0345a1e6 None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:01.827529 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7ee6f0-f80c-4e5d-8f17-751f0345a1e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:01.828012 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7ee6f0-f80c-4e5d-8f17-751f0345a1e6 None admin] PATH_INFO: `/v3/projects` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:01.828993 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e7ee6f0-f80c-4e5d-8f17-751f0345a1e6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:19:01.829494 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e7ee6f0-f80c-4e5d-8f17-751f0345a1e6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:19:01.831327 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e7ee6f0-f80c-4e5d-8f17-751f0345a1e6 None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:19:01.848699 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 352/699] 10.222.0.49 () {56 vars in 1148 bytes} [Wed Jun 15 10:19:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:19:01.850154 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7ee6f0-f80c-4e5d-8f17-751f0345a1e6 None admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:19:01.883302 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:19:01.893350 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 348/700] 10.222.0.49 () {58 vars in 1023 bytes} [Wed Jun 15 10:19:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 10:19:01.894383 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95586fd-af8c-4761-956b-603ca16746d8 None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:19:01.923844 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=100 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:19:01.971316 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7ee6f0-f80c-4e5d-8f17-751f0345a1e6 None admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:19:01.998489 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95586fd-af8c-4761-956b-603ca16746d8 None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:19:02.008713 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b1845e-de3e-4fcd-9f45-cb45b518d741 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:19:02.009519 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b1845e-de3e-4fcd-9f45-cb45b518d741 None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:02.009804 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b1845e-de3e-4fcd-9f45-cb45b518d741 None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:02.010082 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b1845e-de3e-4fcd-9f45-cb45b518d741 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:02.022121 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b1845e-de3e-4fcd-9f45-cb45b518d741 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:19:02.023709 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b1845e-de3e-4fcd-9f45-cb45b518d741 None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:19:02.034750 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e417d518-2639-4b71-8c6f-bd6b004cc251 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:19:02.035382 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e417d518-2639-4b71-8c6f-bd6b004cc251 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:02.035667 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e417d518-2639-4b71-8c6f-bd6b004cc251 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:02.035944 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e417d518-2639-4b71-8c6f-bd6b004cc251 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:02.036770 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e417d518-2639-4b71-8c6f-bd6b004cc251 None admin] RBAC: Adding query filter params (role.id=8051174e7a7947d5b991ebd793c2e543, scope.project.id=559b3e713f5844b6ae37151d56dd419f, user.id=5ac54528370d4747b6e8777fba9d0c82) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:19:02.037080 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e417d518-2639-4b71-8c6f-bd6b004cc251 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:19:02.038503 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e417d518-2639-4b71-8c6f-bd6b004cc251 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:19:02.043760 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application [None req-96b1845e-de3e-4fcd-9f45-cb45b518d741 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 10:19:02.043760 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:19:02.043760 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:19:02.043760 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:19:02.043760 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:19:02.043760 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:19:02.043760 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:19:02.043760 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:19:02.043760 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:19:02.043760 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:19:02.043760 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:19:02.043760 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:19:02.043760 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 10:19:02.043760 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 10:19:02.043760 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 10:19:02.044619 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 10:19:02.044619 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:19:02.044619 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:19:02.044619 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:19:02.044619 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:19:02.044619 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:19:02.044619 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:19:02.044619 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:19:02.044619 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application with Lock( Jun 15 10:19:02.044619 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:19:02.044619 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:19:02.044619 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:19:02.044619 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:19:02.044619 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:19:02.044619 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:19:02.044619 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:19:02.044619 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:19:02.045507 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 10:19:02.045507 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 10:19:02.045507 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 10:19:02.045507 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application Jun 15 10:19:02.045725 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 349/701] 10.222.0.49 () {56 vars in 1374 bytes} [Wed Jun 15 10:19:01 2022] GET /identity/v3/role_assignments?user.id=5ac54528370d4747b6e8777fba9d0c82&scope.project.id=559b3e713f5844b6ae37151d56dd419f&role.id=8051174e7a7947d5b991ebd793c2e543 => generated 259 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:19:02.045725 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e417d518-2639-4b71-8c6f-bd6b004cc251 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:19:02.047238 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 353/702] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Jun 15 10:19:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:19:02.065403 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b1845e-de3e-4fcd-9f45-cb45b518d741 None admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:19:02.186568 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:19:02.267472 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b1845e-de3e-4fcd-9f45-cb45b518d741 None admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:19:02.277347 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e417d518-2639-4b71-8c6f-bd6b004cc251 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:19:02.288926 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=100 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:19:02.318809 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c20bf6c-d640-4448-ab44-29355d8c56c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:19:02.320218 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c20bf6c-d640-4448-ab44-29355d8c56c6 None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:02.320502 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c20bf6c-d640-4448-ab44-29355d8c56c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:02.320784 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c20bf6c-d640-4448-ab44-29355d8c56c6 None admin] PATH_INFO: `/v3/domains` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:02.321514 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c20bf6c-d640-4448-ab44-29355d8c56c6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:19:02.321826 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c20bf6c-d640-4448-ab44-29355d8c56c6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:19:02.323265 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c20bf6c-d640-4448-ab44-29355d8c56c6 None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:19:02.326661 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b57f46e-76aa-4ca2-bbe9-337880358ea5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:19:02.327492 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b57f46e-76aa-4ca2-bbe9-337880358ea5 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:02.327777 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b57f46e-76aa-4ca2-bbe9-337880358ea5 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:02.328051 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b57f46e-76aa-4ca2-bbe9-337880358ea5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:02.341178 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b57f46e-76aa-4ca2-bbe9-337880358ea5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:19:02.342339 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 354/703] 10.222.0.49 () {56 vars in 1109 bytes} [Wed Jun 15 10:19:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:19:02.342639 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b57f46e-76aa-4ca2-bbe9-337880358ea5 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:19:02.344612 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e203bd-1039-4e81-82ea-ed2cb8cbf331 None None] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:02.345115 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e203bd-1039-4e81-82ea-ed2cb8cbf331 None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:02.345910 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e203bd-1039-4e81-82ea-ed2cb8cbf331 None None] PATH_INFO: `/` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:02.347836 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 355/704] 10.222.0.49 () {54 vars in 926 bytes} [Wed Jun 15 10:19:02 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:19:02.348748 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e203bd-1039-4e81-82ea-ed2cb8cbf331 None None] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:19:02.368258 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application [None req-4b57f46e-76aa-4ca2-bbe9-337880358ea5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 10:19:02.368258 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:19:02.368258 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:19:02.368258 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:19:02.368258 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:19:02.368258 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:19:02.368258 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:19:02.368258 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:19:02.368258 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:19:02.368258 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:19:02.368258 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:19:02.368258 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:19:02.368258 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 10:19:02.368258 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 10:19:02.368258 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 10:19:02.369224 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 10:19:02.369224 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:19:02.369224 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:19:02.369224 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:19:02.369224 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:19:02.369224 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:19:02.369224 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:19:02.369224 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:19:02.369224 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application with Lock( Jun 15 10:19:02.369224 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:19:02.369224 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:19:02.369224 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:19:02.369224 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:19:02.369224 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:19:02.369224 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:19:02.369224 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:19:02.369224 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:19:02.370168 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 10:19:02.370168 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 10:19:02.370168 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 10:19:02.370168 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Jun 15 10:19:02.370168 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 350/705] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Jun 15 10:19:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 324 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:19:02.371059 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c85c9f-0621-4236-bb8b-d158d1b31f1c None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:02.371380 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c85c9f-0621-4236-bb8b-d158d1b31f1c None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:02.371663 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c85c9f-0621-4236-bb8b-d158d1b31f1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:02.392959 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-88c85c9f-0621-4236-bb8b-d158d1b31f1c None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:19:02.394142 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-88c85c9f-0621-4236-bb8b-d158d1b31f1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:19:02.475366 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 351/706] 10.222.0.49 () {58 vars in 1023 bytes} [Wed Jun 15 10:19:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 10:19:02.476275 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c85c9f-0621-4236-bb8b-d158d1b31f1c None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:19:02.483134 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e203bd-1039-4e81-82ea-ed2cb8cbf331 None None] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:19:02.525862 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c917fbd0-7d38-4209-872a-8d2ee088da89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:19:02.527601 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c917fbd0-7d38-4209-872a-8d2ee088da89 None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:02.528257 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c917fbd0-7d38-4209-872a-8d2ee088da89 None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:02.529614 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c917fbd0-7d38-4209-872a-8d2ee088da89 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:02.537729 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:19:02.538194 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c917fbd0-7d38-4209-872a-8d2ee088da89 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:19:02.541162 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c917fbd0-7d38-4209-872a-8d2ee088da89 None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:19:02.558909 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application [None req-c917fbd0-7d38-4209-872a-8d2ee088da89 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 10:19:02.558909 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:19:02.558909 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:19:02.558909 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:19:02.558909 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:19:02.558909 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:19:02.558909 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:19:02.558909 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:19:02.558909 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:19:02.558909 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:19:02.558909 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:19:02.558909 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:19:02.558909 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 15 10:19:02.558909 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 15 10:19:02.558909 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 15 10:19:02.559947 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 15 10:19:02.559947 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:19:02.559947 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:19:02.559947 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:19:02.559947 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:19:02.559947 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:19:02.559947 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:19:02.559947 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:19:02.559947 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application with Lock( Jun 15 10:19:02.559947 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:19:02.559947 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:19:02.559947 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:19:02.559947 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:19:02.559947 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:19:02.559947 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:19:02.559947 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:19:02.559947 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:19:02.560870 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 15 10:19:02.560870 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 15 10:19:02.560870 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 15 10:19:02.560870 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 15 10:19:02.560870 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 15 10:19:02.560870 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 15 10:19:02.560870 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 10:19:02.560870 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application Jun 15 10:19:02.560870 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 356/707] 10.222.0.49 () {56 vars in 1111 bytes} [Wed Jun 15 10:19:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:19:02.561876 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c917fbd0-7d38-4209-872a-8d2ee088da89 None admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:19:02.600443 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=100 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:19:02.610954 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c85c9f-0621-4236-bb8b-d158d1b31f1c None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:19:02.647708 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e12928-8dc1-4f0a-8d13-37e3516ac319 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:19:02.648379 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e12928-8dc1-4f0a-8d13-37e3516ac319 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:02.648803 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e12928-8dc1-4f0a-8d13-37e3516ac319 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:02.649025 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e12928-8dc1-4f0a-8d13-37e3516ac319 None admin] PATH_INFO: `/v3/domains` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:02.649753 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e12928-8dc1-4f0a-8d13-37e3516ac319 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:19:02.650069 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e12928-8dc1-4f0a-8d13-37e3516ac319 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:19:02.652177 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e12928-8dc1-4f0a-8d13-37e3516ac319 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:19:02.668865 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 352/708] 10.222.0.49 () {56 vars in 1109 bytes} [Wed Jun 15 10:19:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:19:02.670650 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e12928-8dc1-4f0a-8d13-37e3516ac319 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:19:02.689061 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c917fbd0-7d38-4209-872a-8d2ee088da89 None admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:19:02.729916 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f92a6b-e1be-4bed-b50d-eb1ab3ac7338 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:19:02.730956 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f92a6b-e1be-4bed-b50d-eb1ab3ac7338 None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:02.731330 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f92a6b-e1be-4bed-b50d-eb1ab3ac7338 None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:02.731704 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f92a6b-e1be-4bed-b50d-eb1ab3ac7338 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:02.732444 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f92a6b-e1be-4bed-b50d-eb1ab3ac7338 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:19:02.734062 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f92a6b-e1be-4bed-b50d-eb1ab3ac7338 None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:19:02.749275 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application [None req-c1f92a6b-e1be-4bed-b50d-eb1ab3ac7338 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 15 10:19:02.749275 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:19:02.749275 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 15 10:19:02.749275 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 15 10:19:02.749275 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 15 10:19:02.749275 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 15 10:19:02.749275 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 15 10:19:02.749275 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 10:19:02.749275 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 15 10:19:02.749275 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application Jun 15 10:19:02.749275 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 15 10:19:02.749275 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application Jun 15 10:19:02.749275 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:19:02.749275 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:19:02.749275 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:19:02.749275 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:19:02.750610 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:19:02.750610 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:19:02.750610 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:19:02.750610 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:19:02.750610 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:19:02.750610 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:19:02.750610 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:19:02.750610 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 15 10:19:02.750610 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 15 10:19:02.750610 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 15 10:19:02.750610 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 15 10:19:02.750610 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:19:02.750610 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:19:02.750610 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:19:02.750610 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:19:02.750610 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:19:02.751747 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:19:02.751747 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:19:02.751747 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application with Lock( Jun 15 10:19:02.751747 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:19:02.751747 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:19:02.751747 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:19:02.751747 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:19:02.751747 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:19:02.751747 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:19:02.751747 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:19:02.751747 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:19:02.751747 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 15 10:19:02.751747 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 10:19:02.751747 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 15 10:19:02.751747 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application Jun 15 10:19:02.752681 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 357/709] 10.222.0.49 () {56 vars in 1129 bytes} [Wed Jun 15 10:19:02 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:19:02.753527 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f92a6b-e1be-4bed-b50d-eb1ab3ac7338 None admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:19:02.808624 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e12928-8dc1-4f0a-8d13-37e3516ac319 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:19:02.810312 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:19:02.845728 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44018b0a-9039-4be3-81cd-9490e46fa745 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:19:02.846581 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44018b0a-9039-4be3-81cd-9490e46fa745 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:02.846696 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44018b0a-9039-4be3-81cd-9490e46fa745 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:02.846931 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44018b0a-9039-4be3-81cd-9490e46fa745 None admin] PATH_INFO: `/v3/projects` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:02.847745 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44018b0a-9039-4be3-81cd-9490e46fa745 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:19:02.848048 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44018b0a-9039-4be3-81cd-9490e46fa745 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:19:02.849468 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44018b0a-9039-4be3-81cd-9490e46fa745 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:19:02.858699 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=100 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:19:02.863209 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 353/710] 10.222.0.49 () {56 vars in 1148 bytes} [Wed Jun 15 10:19:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:19:02.864265 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44018b0a-9039-4be3-81cd-9490e46fa745 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:19:02.871271 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f92a6b-e1be-4bed-b50d-eb1ab3ac7338 None admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:19:02.907461 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71783c7b-14ae-4fc8-8d00-8af8ec9223bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:19:02.908127 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71783c7b-14ae-4fc8-8d00-8af8ec9223bc None admin] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:02.908423 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71783c7b-14ae-4fc8-8d00-8af8ec9223bc None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:02.908689 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71783c7b-14ae-4fc8-8d00-8af8ec9223bc None admin] PATH_INFO: `/v3/users` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:02.909390 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71783c7b-14ae-4fc8-8d00-8af8ec9223bc None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:19:02.910835 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71783c7b-14ae-4fc8-8d00-8af8ec9223bc None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:19:02.933061 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 358/711] 10.222.0.49 () {60 vars in 1132 bytes} [Wed Jun 15 10:19:02 2022] POST /identity/v3/users => generated 254 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 10:19:02.934426 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da7cfcc-50d6-4d10-b11b-e7491a24835b None None] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:02.934716 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da7cfcc-50d6-4d10-b11b-e7491a24835b None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:02.934990 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da7cfcc-50d6-4d10-b11b-e7491a24835b None None] PATH_INFO: `/` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:02.939213 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 359/712] 10.222.0.49 () {54 vars in 926 bytes} [Wed Jun 15 10:19:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:19:02.939213 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da7cfcc-50d6-4d10-b11b-e7491a24835b None None] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:19:02.989981 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44018b0a-9039-4be3-81cd-9490e46fa745 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:19:03.031422 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217c45f7-c5ec-40ea-86c1-2dc4ec54ccdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:19:03.032546 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217c45f7-c5ec-40ea-86c1-2dc4ec54ccdf None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:03.033117 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217c45f7-c5ec-40ea-86c1-2dc4ec54ccdf None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:03.033568 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217c45f7-c5ec-40ea-86c1-2dc4ec54ccdf None admin] PATH_INFO: `/v3/services` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:03.034558 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-217c45f7-c5ec-40ea-86c1-2dc4ec54ccdf None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:19:03.035068 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-217c45f7-c5ec-40ea-86c1-2dc4ec54ccdf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:19:03.036776 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-217c45f7-c5ec-40ea-86c1-2dc4ec54ccdf None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:19:03.044902 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 354/713] 10.222.0.49 () {56 vars in 1124 bytes} [Wed Jun 15 10:19:02 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:19:03.046690 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2066913-2b5c-4f7f-9329-b3ad15f50bdf None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:03.047186 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2066913-2b5c-4f7f-9329-b3ad15f50bdf None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:03.047656 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2066913-2b5c-4f7f-9329-b3ad15f50bdf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:03.066951 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-b2066913-2b5c-4f7f-9329-b3ad15f50bdf None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:19:03.068598 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-b2066913-2b5c-4f7f-9329-b3ad15f50bdf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:19:03.081010 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da7cfcc-50d6-4d10-b11b-e7491a24835b None None] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:19:03.093686 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:19:03.132170 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f2a6c9-f89c-4fac-9452-b3aab5330e18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:19:03.133106 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f2a6c9-f89c-4fac-9452-b3aab5330e18 None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:03.133836 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f2a6c9-f89c-4fac-9452-b3aab5330e18 None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:03.134788 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f2a6c9-f89c-4fac-9452-b3aab5330e18 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:03.136338 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f2a6c9-f89c-4fac-9452-b3aab5330e18 None admin] RBAC: Adding query filter params (role.id=16e76452bb5d4ce68a119ad3c9bde41c, scope.project.id=559b3e713f5844b6ae37151d56dd419f, user.id=3a69f99210444f9696e8b22615fb4a41) {{(pid=80119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:19:03.136778 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f2a6c9-f89c-4fac-9452-b3aab5330e18 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:19:03.138759 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f2a6c9-f89c-4fac-9452-b3aab5330e18 None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:19:03.145611 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 360/714] 10.222.0.49 () {56 vars in 1374 bytes} [Wed Jun 15 10:19:02 2022] GET /identity/v3/role_assignments?user.id=3a69f99210444f9696e8b22615fb4a41&scope.project.id=559b3e713f5844b6ae37151d56dd419f&role.id=16e76452bb5d4ce68a119ad3c9bde41c => generated 608 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:19:03.152053 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f2a6c9-f89c-4fac-9452-b3aab5330e18 None admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:19:03.158067 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 355/715] 10.222.0.49 () {58 vars in 1023 bytes} [Wed Jun 15 10:19:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 10:19:03.180177 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:19:03.187074 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2066913-2b5c-4f7f-9329-b3ad15f50bdf None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:19:03.308373 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f2a6c9-f89c-4fac-9452-b3aab5330e18 None admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:19:03.318516 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2066913-2b5c-4f7f-9329-b3ad15f50bdf None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:19:03.357131 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59f5578-a7e3-40ee-8387-e4c455d875d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:19:03.358082 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb58d5b-f1d7-4fbd-96b7-044b3af297ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:19:03.358233 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59f5578-a7e3-40ee-8387-e4c455d875d2 None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:03.358400 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59f5578-a7e3-40ee-8387-e4c455d875d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:03.358678 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59f5578-a7e3-40ee-8387-e4c455d875d2 None admin] PATH_INFO: `/v3/services` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:03.362440 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb58d5b-f1d7-4fbd-96b7-044b3af297ea None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:03.362559 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e59f5578-a7e3-40ee-8387-e4c455d875d2 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:19:03.362637 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb58d5b-f1d7-4fbd-96b7-044b3af297ea None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:03.362707 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e59f5578-a7e3-40ee-8387-e4c455d875d2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:19:03.362795 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb58d5b-f1d7-4fbd-96b7-044b3af297ea None admin] PATH_INFO: `/v3/services/network` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:03.362795 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb58d5b-f1d7-4fbd-96b7-044b3af297ea None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:19:03.362921 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e59f5578-a7e3-40ee-8387-e4c455d875d2 None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:19:03.362989 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb58d5b-f1d7-4fbd-96b7-044b3af297ea None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:19:03.368474 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 361/716] 10.222.0.49 () {56 vars in 1124 bytes} [Wed Jun 15 10:19:03 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:19:03.379863 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application [None req-ddb58d5b-f1d7-4fbd-96b7-044b3af297ea None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jun 15 10:19:03.379863 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:19:03.379863 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 15 10:19:03.379863 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 15 10:19:03.379863 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 15 10:19:03.379863 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 15 10:19:03.379863 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 15 10:19:03.379863 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 10:19:03.379863 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 15 10:19:03.379863 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Jun 15 10:19:03.379863 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 15 10:19:03.379863 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Jun 15 10:19:03.379863 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:19:03.379863 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:19:03.379863 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:19:03.379863 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:19:03.379863 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:19:03.381101 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:19:03.381101 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:19:03.381101 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:19:03.381101 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:19:03.381101 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:19:03.381101 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:19:03.381101 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 15 10:19:03.381101 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 15 10:19:03.381101 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 15 10:19:03.381101 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 15 10:19:03.381101 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:19:03.381101 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:19:03.381101 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:19:03.381101 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:19:03.381101 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:19:03.381101 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:19:03.382129 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:19:03.382129 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application with Lock( Jun 15 10:19:03.382129 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:19:03.382129 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:19:03.382129 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:19:03.382129 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:19:03.382129 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:19:03.382129 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:19:03.382129 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:19:03.382129 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:19:03.382129 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 15 10:19:03.382129 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 10:19:03.382129 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 15 10:19:03.382129 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Jun 15 10:19:03.387182 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 356/717] 10.222.0.49 () {56 vars in 1111 bytes} [Wed Jun 15 10:19:03 2022] GET /identity/v3/services/network => generated 88 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:19:03.389659 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59f5578-a7e3-40ee-8387-e4c455d875d2 None admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:19:03.484182 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=100 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:19:03.511516 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59f5578-a7e3-40ee-8387-e4c455d875d2 None admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:19:03.553754 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05fc92dc-fc5a-46f0-8088-ad38006c1ebd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:19:03.554441 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fc92dc-fc5a-46f0-8088-ad38006c1ebd None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:03.554730 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fc92dc-fc5a-46f0-8088-ad38006c1ebd None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:03.555017 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fc92dc-fc5a-46f0-8088-ad38006c1ebd None admin] PATH_INFO: `/v3/services` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:03.555764 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05fc92dc-fc5a-46f0-8088-ad38006c1ebd None admin] RBAC: Adding query filter params (name=network) {{(pid=80119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:19:03.556074 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05fc92dc-fc5a-46f0-8088-ad38006c1ebd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:19:03.557499 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05fc92dc-fc5a-46f0-8088-ad38006c1ebd None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:19:03.562639 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 362/718] 10.222.0.49 () {56 vars in 1112 bytes} [Wed Jun 15 10:19:03 2022] GET /identity/v3/services?name=network => generated 124 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:19:03.568046 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb58d5b-f1d7-4fbd-96b7-044b3af297ea None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:19:03.601969 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:19:03.702506 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb58d5b-f1d7-4fbd-96b7-044b3af297ea None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:19:03.742616 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83dafbe-781d-4dfa-a524-4e1628724a2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:19:03.743349 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83dafbe-781d-4dfa-a524-4e1628724a2b None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:03.743637 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83dafbe-781d-4dfa-a524-4e1628724a2b None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:03.743912 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83dafbe-781d-4dfa-a524-4e1628724a2b None admin] PATH_INFO: `/v3/services` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:03.744626 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e83dafbe-781d-4dfa-a524-4e1628724a2b None admin] RBAC: Adding query filter params (type=network) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:19:03.744981 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e83dafbe-781d-4dfa-a524-4e1628724a2b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:19:03.746350 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e83dafbe-781d-4dfa-a524-4e1628724a2b None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:19:03.752590 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 357/719] 10.222.0.49 () {56 vars in 1112 bytes} [Wed Jun 15 10:19:03 2022] GET /identity/v3/services?type=network => generated 124 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:19:04.440861 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d921c68b-77a3-4527-bd5c-b3e1a1c058a7 None None] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:04.441383 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d921c68b-77a3-4527-bd5c-b3e1a1c058a7 None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:04.441463 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d921c68b-77a3-4527-bd5c-b3e1a1c058a7 None None] PATH_INFO: `/` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:04.442942 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 363/720] 10.222.0.49 () {54 vars in 926 bytes} [Wed Jun 15 10:19:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:19:04.448933 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0b1702-461a-4449-bdba-0bebf1cb901c None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:04.449735 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0b1702-461a-4449-bdba-0bebf1cb901c None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:04.450219 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0b1702-461a-4449-bdba-0bebf1cb901c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:04.467342 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-8e0b1702-461a-4449-bdba-0bebf1cb901c None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:19:04.468872 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-8e0b1702-461a-4449-bdba-0bebf1cb901c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:19:04.542006 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 358/721] 10.222.0.49 () {58 vars in 1023 bytes} [Wed Jun 15 10:19:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 10:19:04.562159 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d921c68b-77a3-4527-bd5c-b3e1a1c058a7 None None] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:19:04.703809 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d921c68b-77a3-4527-bd5c-b3e1a1c058a7 None None] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:19:04.743246 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60900d68-2c7f-40e5-9977-1d3e8ae407e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:19:04.744465 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60900d68-2c7f-40e5-9977-1d3e8ae407e8 None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:04.745520 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60900d68-2c7f-40e5-9977-1d3e8ae407e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:04.746004 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60900d68-2c7f-40e5-9977-1d3e8ae407e8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:04.764170 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60900d68-2c7f-40e5-9977-1d3e8ae407e8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 10:19:04.777842 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application [None req-60900d68-2c7f-40e5-9977-1d3e8ae407e8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 10:19:04.777842 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:19:04.777842 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:19:04.777842 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:19:04.777842 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:19:04.777842 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:19:04.777842 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:19:04.777842 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:19:04.777842 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:19:04.777842 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:19:04.777842 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:19:04.777842 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:19:04.777842 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 10:19:04.777842 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 10:19:04.777842 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 10:19:04.777842 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 10:19:04.778825 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 10:19:04.778825 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 10:19:04.778825 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 10:19:04.778825 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 10:19:04.778825 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:19:04.778825 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:19:04.778825 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:19:04.778825 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:19:04.778825 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:19:04.778825 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:19:04.778825 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:19:04.778825 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application with Lock( Jun 15 10:19:04.778825 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:19:04.778825 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:19:04.778825 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:19:04.778825 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:19:04.784130 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:19:04.784130 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:19:04.784130 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:19:04.784130 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:19:04.784130 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 10:19:04.784130 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 10:19:04.784130 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 10:19:04.784130 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 10:19:04.784130 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 10:19:04.784130 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 10:19:04.784130 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 10:19:04.784130 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application Jun 15 10:19:04.786404 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 364/722] 10.222.0.49 () {56 vars in 1096 bytes} [Wed Jun 15 10:19:04 2022] GET /identity/v3/roles/admin => generated 83 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:19:04.796020 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0b1702-461a-4449-bdba-0bebf1cb901c None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:19:04.823443 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:19:04.942749 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0b1702-461a-4449-bdba-0bebf1cb901c None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:19:04.989206 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89147d5d-384d-4ad2-906c-f50ddf352afa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:19:04.989798 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89147d5d-384d-4ad2-906c-f50ddf352afa None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:04.990070 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89147d5d-384d-4ad2-906c-f50ddf352afa None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:04.990365 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89147d5d-384d-4ad2-906c-f50ddf352afa None admin] PATH_INFO: `/v3/roles` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:04.991139 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89147d5d-384d-4ad2-906c-f50ddf352afa None admin] RBAC: Adding query filter params (name=admin) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:19:04.991422 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89147d5d-384d-4ad2-906c-f50ddf352afa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:19:04.992793 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89147d5d-384d-4ad2-906c-f50ddf352afa None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:19:05.002053 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 359/723] 10.222.0.49 () {56 vars in 1099 bytes} [Wed Jun 15 10:19:04 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:19:05.007152 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60900d68-2c7f-40e5-9977-1d3e8ae407e8 None admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:19:05.015413 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=100 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:19:05.150879 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60900d68-2c7f-40e5-9977-1d3e8ae407e8 None admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:19:05.195217 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6187ac90-5053-4101-a6ab-5e1d0347c2ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:19:05.196414 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6187ac90-5053-4101-a6ab-5e1d0347c2ea None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:05.196924 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6187ac90-5053-4101-a6ab-5e1d0347c2ea None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:05.197409 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6187ac90-5053-4101-a6ab-5e1d0347c2ea None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:05.216024 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6187ac90-5053-4101-a6ab-5e1d0347c2ea None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:19:05.217503 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6187ac90-5053-4101-a6ab-5e1d0347c2ea None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:19:05.235454 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application [None req-6187ac90-5053-4101-a6ab-5e1d0347c2ea None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 10:19:05.235454 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:19:05.235454 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:19:05.235454 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:19:05.235454 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:19:05.235454 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:19:05.235454 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:19:05.235454 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:19:05.235454 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:19:05.235454 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:19:05.235454 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:19:05.235454 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:19:05.235454 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 10:19:05.235454 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 10:19:05.235454 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 10:19:05.236356 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 10:19:05.236356 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:19:05.236356 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:19:05.236356 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:19:05.236356 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:19:05.236356 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:19:05.236356 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:19:05.236356 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:19:05.236356 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application with Lock( Jun 15 10:19:05.236356 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:19:05.236356 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:19:05.236356 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:19:05.236356 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:19:05.236356 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:19:05.236356 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:19:05.236356 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:19:05.236356 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:19:05.237920 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 10:19:05.237920 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 10:19:05.237920 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 10:19:05.237920 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application Jun 15 10:19:05.237920 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 365/724] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Jun 15 10:19:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:19:05.242197 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89147d5d-384d-4ad2-906c-f50ddf352afa None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:19:05.372402 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:19:05.399914 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89147d5d-384d-4ad2-906c-f50ddf352afa None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:19:05.447324 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8b1eae-a91e-4bad-87e3-720e885391d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:19:05.448540 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8b1eae-a91e-4bad-87e3-720e885391d4 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:05.448828 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8b1eae-a91e-4bad-87e3-720e885391d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:05.449104 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8b1eae-a91e-4bad-87e3-720e885391d4 None admin] PATH_INFO: `/v3/domains` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:05.449834 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c8b1eae-a91e-4bad-87e3-720e885391d4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:19:05.450140 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c8b1eae-a91e-4bad-87e3-720e885391d4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:19:05.451547 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c8b1eae-a91e-4bad-87e3-720e885391d4 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:19:05.468519 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 360/725] 10.222.0.49 () {56 vars in 1109 bytes} [Wed Jun 15 10:19:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:19:05.473747 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6187ac90-5053-4101-a6ab-5e1d0347c2ea None admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:19:05.537038 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=100 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:19:05.590198 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6187ac90-5053-4101-a6ab-5e1d0347c2ea None admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:19:05.634190 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7e55d9-6d9a-4181-bb1c-911922fa2070 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:19:05.634993 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7e55d9-6d9a-4181-bb1c-911922fa2070 None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:05.635824 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7e55d9-6d9a-4181-bb1c-911922fa2070 None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:05.636111 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7e55d9-6d9a-4181-bb1c-911922fa2070 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:05.646963 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e7e55d9-6d9a-4181-bb1c-911922fa2070 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:19:05.648513 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e7e55d9-6d9a-4181-bb1c-911922fa2070 None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:19:05.667005 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application [None req-8e7e55d9-6d9a-4181-bb1c-911922fa2070 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 15 10:19:05.667005 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:19:05.667005 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:19:05.667005 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:19:05.667005 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:19:05.667005 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:19:05.667005 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:19:05.667005 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:19:05.667005 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:19:05.667005 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:19:05.667005 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:19:05.667005 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:19:05.667005 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 15 10:19:05.667005 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 15 10:19:05.667005 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 15 10:19:05.667005 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 15 10:19:05.669330 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:19:05.669330 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:19:05.669330 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 15 10:19:05.669330 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 10:19:05.669330 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 15 10:19:05.669330 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 10:19:05.669330 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:19:05.669330 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:19:05.669330 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:19:05.669330 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:19:05.669330 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:19:05.669330 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application with Lock( Jun 15 10:19:05.669330 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:19:05.669330 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:19:05.669330 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:19:05.669330 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:19:05.669330 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:19:05.669330 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:19:05.671670 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:19:05.671670 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:19:05.671670 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 15 10:19:05.671670 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 15 10:19:05.671670 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 15 10:19:05.671670 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 15 10:19:05.671670 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 15 10:19:05.671670 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 15 10:19:05.671670 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 15 10:19:05.671670 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application Jun 15 10:19:05.673731 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 366/726] 10.222.0.49 () {56 vars in 1093 bytes} [Wed Jun 15 10:19:05 2022] GET /identity/v3/users/nova => generated 82 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:19:05.678435 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8b1eae-a91e-4bad-87e3-720e885391d4 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:19:05.731778 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06b0416-cb28-465e-afd9-9520f2b04244 None None] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:05.732486 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06b0416-cb28-465e-afd9-9520f2b04244 None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:05.733011 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06b0416-cb28-465e-afd9-9520f2b04244 None None] PATH_INFO: `/` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:05.735828 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 367/727] 10.222.0.49 () {54 vars in 926 bytes} [Wed Jun 15 10:19:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 15 10:19:05.739957 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3ef713-45d1-4e59-9196-96b758a656e0 None None] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:05.740485 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3ef713-45d1-4e59-9196-96b758a656e0 None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:05.740995 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3ef713-45d1-4e59-9196-96b758a656e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:05.779833 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.auth.core [None req-3b3ef713-45d1-4e59-9196-96b758a656e0 None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:19:05.779833 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-3b3ef713-45d1-4e59-9196-96b758a656e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:19:05.803096 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8b1eae-a91e-4bad-87e3-720e885391d4 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:19:05.839544 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd857c3-85ce-4289-879f-204b368fddb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:19:05.840242 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd857c3-85ce-4289-879f-204b368fddb7 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:05.840527 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd857c3-85ce-4289-879f-204b368fddb7 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:05.840832 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd857c3-85ce-4289-879f-204b368fddb7 None admin] PATH_INFO: `/v3/users` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:05.841642 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd857c3-85ce-4289-879f-204b368fddb7 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:19:05.841945 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd857c3-85ce-4289-879f-204b368fddb7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:19:05.843409 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd857c3-85ce-4289-879f-204b368fddb7 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:19:05.847426 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:19:05.871116 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 361/728] 10.222.0.49 () {56 vars in 1133 bytes} [Wed Jun 15 10:19:05 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:19:05.874571 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd857c3-85ce-4289-879f-204b368fddb7 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:19:05.945697 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 368/729] 10.222.0.49 () {58 vars in 1023 bytes} [Wed Jun 15 10:19:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 15 10:19:05.988621 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd857c3-85ce-4289-879f-204b368fddb7 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:19:05.989729 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3ef713-45d1-4e59-9196-96b758a656e0 None None] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:19:06.026227 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806de9f2-4e32-4fbf-b799-e99389ec8732 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:19:06.027038 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806de9f2-4e32-4fbf-b799-e99389ec8732 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:06.027352 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806de9f2-4e32-4fbf-b799-e99389ec8732 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:06.027629 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806de9f2-4e32-4fbf-b799-e99389ec8732 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:06.038028 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-806de9f2-4e32-4fbf-b799-e99389ec8732 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:19:06.039532 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-806de9f2-4e32-4fbf-b799-e99389ec8732 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:19:06.058482 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application [None req-806de9f2-4e32-4fbf-b799-e99389ec8732 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 10:19:06.058482 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:19:06.058482 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:19:06.058482 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:19:06.058482 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:19:06.058482 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:19:06.058482 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:19:06.058482 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:19:06.058482 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:19:06.058482 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:19:06.058482 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:19:06.058482 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:19:06.058482 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 10:19:06.058482 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 10:19:06.058482 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 10:19:06.059472 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 10:19:06.059472 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:19:06.059472 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:19:06.059472 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:19:06.059472 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:19:06.059472 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:19:06.059472 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:19:06.059472 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:19:06.059472 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application with Lock( Jun 15 10:19:06.059472 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:19:06.059472 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:19:06.059472 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:19:06.059472 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:19:06.059472 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:19:06.059472 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:19:06.059472 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:19:06.059472 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:19:06.060893 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 10:19:06.060893 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 10:19:06.060893 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 10:19:06.060893 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Jun 15 10:19:06.060893 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 362/730] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Jun 15 10:19:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 10:19:06.060893 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120862da-199a-4a74-b972-e04c3e714e67 None None] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:06.061263 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120862da-199a-4a74-b972-e04c3e714e67 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:06.061561 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120862da-199a-4a74-b972-e04c3e714e67 None None] PATH_INFO: `/` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:06.062968 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 363/731] 10.222.0.49 () {54 vars in 926 bytes} [Wed Jun 15 10:19:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:19:06.068390 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cc7ebc-4044-4de6-9fd6-668410c4ffd2 None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:06.068783 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cc7ebc-4044-4de6-9fd6-668410c4ffd2 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:06.069160 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cc7ebc-4044-4de6-9fd6-668410c4ffd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:06.076973 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=100 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:19:06.088221 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-d2cc7ebc-4044-4de6-9fd6-668410c4ffd2 None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:19:06.089578 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-d2cc7ebc-4044-4de6-9fd6-668410c4ffd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:19:06.134371 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:19:06.144169 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3ef713-45d1-4e59-9196-96b758a656e0 None None] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:19:06.161302 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 364/732] 10.222.0.49 () {58 vars in 1023 bytes} [Wed Jun 15 10:19:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 15 10:19:06.162256 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2cc7ebc-4044-4de6-9fd6-668410c4ffd2 None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:19:06.188290 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b27eef-b8d9-448c-a853-f619646dfe8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:19:06.189730 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b27eef-b8d9-448c-a853-f619646dfe8a None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:06.190471 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b27eef-b8d9-448c-a853-f619646dfe8a None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:06.190985 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b27eef-b8d9-448c-a853-f619646dfe8a None admin] PATH_INFO: `/v3/roles/service` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:06.202281 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b27eef-b8d9-448c-a853-f619646dfe8a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 10:19:06.210173 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application [None req-10b27eef-b8d9-448c-a853-f619646dfe8a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 15 10:19:06.210173 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:19:06.210173 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:19:06.210173 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:19:06.210173 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:19:06.210173 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:19:06.210173 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:19:06.210173 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:19:06.210173 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:19:06.210173 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:19:06.210173 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:19:06.210173 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:19:06.210173 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 10:19:06.210173 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 10:19:06.210173 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 10:19:06.210173 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 10:19:06.211169 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 10:19:06.211169 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 10:19:06.211169 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 10:19:06.211169 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 10:19:06.211169 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:19:06.211169 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:19:06.211169 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:19:06.211169 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:19:06.211169 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:19:06.211169 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:19:06.211169 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:19:06.211169 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application with Lock( Jun 15 10:19:06.211169 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:19:06.211169 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:19:06.211169 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:19:06.211169 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:19:06.212223 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:19:06.212223 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:19:06.212223 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:19:06.212223 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:19:06.212223 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 10:19:06.212223 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 10:19:06.212223 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 10:19:06.212223 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 10:19:06.212223 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 10:19:06.212223 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 10:19:06.212223 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 15 10:19:06.212223 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application Jun 15 10:19:06.216012 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 369/733] 10.222.0.49 () {56 vars in 1102 bytes} [Wed Jun 15 10:19:05 2022] GET /identity/v3/roles/service => generated 85 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:19:06.217843 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf1161a-0fc7-4cdd-be89-4fc0178f695a None None] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:06.218145 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf1161a-0fc7-4cdd-be89-4fc0178f695a None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:06.218429 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf1161a-0fc7-4cdd-be89-4fc0178f695a None None] PATH_INFO: `/` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:06.219894 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 370/734] 10.222.0.49 () {54 vars in 926 bytes} [Wed Jun 15 10:19:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:19:06.220539 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf1161a-0fc7-4cdd-be89-4fc0178f695a None None] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:19:06.275003 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2cc7ebc-4044-4de6-9fd6-668410c4ffd2 None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:19:06.312759 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c88a7d-6400-4601-967d-590b04f17e19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:19:06.313471 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c88a7d-6400-4601-967d-590b04f17e19 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:06.313760 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c88a7d-6400-4601-967d-590b04f17e19 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:06.314038 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c88a7d-6400-4601-967d-590b04f17e19 None admin] PATH_INFO: `/v3/domains` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:06.314825 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c88a7d-6400-4601-967d-590b04f17e19 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:19:06.315164 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c88a7d-6400-4601-967d-590b04f17e19 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:19:06.316822 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c88a7d-6400-4601-967d-590b04f17e19 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:19:06.327057 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf1161a-0fc7-4cdd-be89-4fc0178f695a None None] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:19:06.329387 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=100 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:19:06.334574 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 365/735] 10.222.0.49 () {56 vars in 1109 bytes} [Wed Jun 15 10:19:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:19:06.334574 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c88a7d-6400-4601-967d-590b04f17e19 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:19:06.365642 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c722c8dd-27e2-48e7-8a24-6b56a2aecf65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:19:06.366451 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c722c8dd-27e2-48e7-8a24-6b56a2aecf65 None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:06.366737 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c722c8dd-27e2-48e7-8a24-6b56a2aecf65 None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:06.367017 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c722c8dd-27e2-48e7-8a24-6b56a2aecf65 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:06.376250 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:19:06.378287 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c722c8dd-27e2-48e7-8a24-6b56a2aecf65 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 10:19:06.388418 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application [None req-c722c8dd-27e2-48e7-8a24-6b56a2aecf65 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 10:19:06.388418 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:19:06.388418 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:19:06.388418 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:19:06.388418 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:19:06.388418 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:19:06.388418 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:19:06.388418 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:19:06.388418 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:19:06.388418 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:19:06.388418 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:19:06.388418 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:19:06.388418 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 10:19:06.388418 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 10:19:06.388418 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 10:19:06.388418 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 10:19:06.389362 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 10:19:06.389362 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 10:19:06.389362 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 10:19:06.389362 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 10:19:06.389362 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:19:06.389362 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:19:06.389362 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:19:06.389362 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:19:06.389362 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:19:06.389362 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:19:06.389362 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:19:06.389362 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application with Lock( Jun 15 10:19:06.389362 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:19:06.389362 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:19:06.389362 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:19:06.389362 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:19:06.390245 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:19:06.390245 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:19:06.390245 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:19:06.390245 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:19:06.390245 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 10:19:06.390245 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 10:19:06.390245 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 10:19:06.390245 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 10:19:06.390245 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 10:19:06.390245 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 10:19:06.390245 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 10:19:06.390245 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application Jun 15 10:19:06.390245 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 371/736] 10.222.0.49 () {56 vars in 1096 bytes} [Wed Jun 15 10:19:06 2022] GET /identity/v3/roles/admin => generated 83 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:19:06.391620 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e325592b-491e-43a7-b7bd-928e35392cfe None None] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:06.391620 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e325592b-491e-43a7-b7bd-928e35392cfe None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:06.391620 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e325592b-491e-43a7-b7bd-928e35392cfe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:06.411883 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.auth.core [None req-e325592b-491e-43a7-b7bd-928e35392cfe None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:19:06.413649 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-e325592b-491e-43a7-b7bd-928e35392cfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:19:06.450234 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c88a7d-6400-4601-967d-590b04f17e19 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:19:06.495272 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 372/737] 10.222.0.49 () {58 vars in 1023 bytes} [Wed Jun 15 10:19:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 10:19:06.495272 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e325592b-491e-43a7-b7bd-928e35392cfe None None] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:19:06.498644 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6fe0f3-d20e-4a91-9dd8-e4629bf44664 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:19:06.499423 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6fe0f3-d20e-4a91-9dd8-e4629bf44664 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:06.499781 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6fe0f3-d20e-4a91-9dd8-e4629bf44664 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:06.500127 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6fe0f3-d20e-4a91-9dd8-e4629bf44664 None admin] PATH_INFO: `/v3/roles` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:06.501122 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6fe0f3-d20e-4a91-9dd8-e4629bf44664 None admin] RBAC: Adding query filter params (name=service) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:19:06.501501 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6fe0f3-d20e-4a91-9dd8-e4629bf44664 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:19:06.503155 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6fe0f3-d20e-4a91-9dd8-e4629bf44664 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:19:06.512672 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 366/738] 10.222.0.49 () {56 vars in 1103 bytes} [Wed Jun 15 10:19:06 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:19:06.513582 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6fe0f3-d20e-4a91-9dd8-e4629bf44664 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:19:06.600616 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=100 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:19:06.609635 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e325592b-491e-43a7-b7bd-928e35392cfe None None] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:19:06.619692 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:19:06.624860 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6fe0f3-d20e-4a91-9dd8-e4629bf44664 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:19:06.647220 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b118dd86-3c3e-4078-ae02-aeb87fa2c4c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:19:06.648032 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b118dd86-3c3e-4078-ae02-aeb87fa2c4c5 None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:06.648321 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b118dd86-3c3e-4078-ae02-aeb87fa2c4c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:06.648600 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b118dd86-3c3e-4078-ae02-aeb87fa2c4c5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:06.656194 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b118dd86-3c3e-4078-ae02-aeb87fa2c4c5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:19:06.657648 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b118dd86-3c3e-4078-ae02-aeb87fa2c4c5 None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:19:06.660666 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93d1a5b-6498-4020-ad41-58a75397983e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:19:06.661334 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93d1a5b-6498-4020-ad41-58a75397983e None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:06.661622 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93d1a5b-6498-4020-ad41-58a75397983e None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:06.661900 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93d1a5b-6498-4020-ad41-58a75397983e None admin] PATH_INFO: `/v3/roles` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:06.662643 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d93d1a5b-6498-4020-ad41-58a75397983e None admin] RBAC: Adding query filter params (name=admin) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:19:06.662957 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d93d1a5b-6498-4020-ad41-58a75397983e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:19:06.664330 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d93d1a5b-6498-4020-ad41-58a75397983e None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:19:06.671690 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application [None req-b118dd86-3c3e-4078-ae02-aeb87fa2c4c5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 10:19:06.671690 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:19:06.671690 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:19:06.671690 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:19:06.671690 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:19:06.671690 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:19:06.671690 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:19:06.671690 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:19:06.671690 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:19:06.671690 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:19:06.671690 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:19:06.671690 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:19:06.671690 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 15 10:19:06.671690 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 15 10:19:06.671690 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 15 10:19:06.672567 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 15 10:19:06.672567 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:19:06.672567 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:19:06.672567 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:19:06.672567 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:19:06.672567 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:19:06.672567 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:19:06.672567 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:19:06.672567 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application with Lock( Jun 15 10:19:06.672567 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:19:06.672567 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:19:06.672567 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:19:06.672567 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:19:06.672567 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:19:06.672567 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:19:06.672567 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:19:06.672567 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:19:06.678392 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 15 10:19:06.678392 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 15 10:19:06.678392 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 15 10:19:06.678392 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 15 10:19:06.678392 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 15 10:19:06.678392 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 15 10:19:06.678392 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 10:19:06.678392 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application Jun 15 10:19:06.678829 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 367/739] 10.222.0.49 () {56 vars in 1099 bytes} [Wed Jun 15 10:19:06 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:19:06.678903 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 373/740] 10.222.0.49 () {56 vars in 1111 bytes} [Wed Jun 15 10:19:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:19:06.678972 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93d1a5b-6498-4020-ad41-58a75397983e None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:19:06.679041 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b118dd86-3c3e-4078-ae02-aeb87fa2c4c5 None admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:19:06.775616 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93d1a5b-6498-4020-ad41-58a75397983e None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:19:06.777455 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b118dd86-3c3e-4078-ae02-aeb87fa2c4c5 None admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:19:06.809950 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ecb47b3-f532-4fca-be3f-d054cad4e87c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:19:06.810754 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecb47b3-f532-4fca-be3f-d054cad4e87c None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:06.811049 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecb47b3-f532-4fca-be3f-d054cad4e87c None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:06.811359 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecb47b3-f532-4fca-be3f-d054cad4e87c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:06.814996 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99b10af-404e-43e4-801e-cab6d7942542 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:19:06.815685 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99b10af-404e-43e4-801e-cab6d7942542 None admin] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:06.815971 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99b10af-404e-43e4-801e-cab6d7942542 None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:06.816249 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99b10af-404e-43e4-801e-cab6d7942542 None admin] PATH_INFO: `/v3/services` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:06.816961 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99b10af-404e-43e4-801e-cab6d7942542 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:19:06.818341 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99b10af-404e-43e4-801e-cab6d7942542 None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:19:06.821693 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ecb47b3-f532-4fca-be3f-d054cad4e87c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:19:06.823332 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ecb47b3-f532-4fca-be3f-d054cad4e87c None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:19:06.832258 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 374/741] 10.222.0.49 () {60 vars in 1141 bytes} [Wed Jun 15 10:19:06 2022] POST /identity/v3/services => generated 251 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 10:19:06.833602 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b6855b-c9c5-4fff-ad4b-b25394af27de None None] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:06.833899 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b6855b-c9c5-4fff-ad4b-b25394af27de None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:06.834177 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b6855b-c9c5-4fff-ad4b-b25394af27de None None] PATH_INFO: `/` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:06.839145 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 375/742] 10.222.0.49 () {54 vars in 926 bytes} [Wed Jun 15 10:19:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:19:06.839145 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b6855b-c9c5-4fff-ad4b-b25394af27de None None] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:19:06.842673 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application [None req-0ecb47b3-f532-4fca-be3f-d054cad4e87c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 10:19:06.842673 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:19:06.842673 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:19:06.842673 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:19:06.842673 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:19:06.842673 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:19:06.842673 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:19:06.842673 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:19:06.842673 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:19:06.842673 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:19:06.842673 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:19:06.842673 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:19:06.842673 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 10:19:06.842673 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 10:19:06.842673 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 10:19:06.843812 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 10:19:06.843812 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:19:06.843812 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:19:06.843812 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:19:06.843812 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:19:06.843812 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:19:06.843812 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:19:06.843812 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:19:06.843812 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application with Lock( Jun 15 10:19:06.843812 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:19:06.843812 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:19:06.843812 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:19:06.843812 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:19:06.843812 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:19:06.843812 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:19:06.843812 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:19:06.843812 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:19:06.844922 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 10:19:06.844922 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 10:19:06.844922 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 10:19:06.844922 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Jun 15 10:19:06.844922 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 368/743] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Jun 15 10:19:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:19:06.845442 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ecb47b3-f532-4fca-be3f-d054cad4e87c None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:19:06.866265 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:19:06.872667 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:19:06.971565 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ecb47b3-f532-4fca-be3f-d054cad4e87c None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:19:06.979829 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b6855b-c9c5-4fff-ad4b-b25394af27de None None] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:19:07.018364 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98715da3-cb59-48d4-bb3e-24cea238ff81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:19:07.018807 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2a32a8-9bf8-4213-beea-74dbbef5052f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:19:07.019614 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98715da3-cb59-48d4-bb3e-24cea238ff81 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:07.019737 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2a32a8-9bf8-4213-beea-74dbbef5052f None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:07.019921 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2a32a8-9bf8-4213-beea-74dbbef5052f None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:07.020155 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98715da3-cb59-48d4-bb3e-24cea238ff81 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:07.020278 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2a32a8-9bf8-4213-beea-74dbbef5052f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:07.020434 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98715da3-cb59-48d4-bb3e-24cea238ff81 None admin] PATH_INFO: `/v3/projects` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:07.021232 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98715da3-cb59-48d4-bb3e-24cea238ff81 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:19:07.021549 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98715da3-cb59-48d4-bb3e-24cea238ff81 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:19:07.022978 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98715da3-cb59-48d4-bb3e-24cea238ff81 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:19:07.033988 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f2a32a8-9bf8-4213-beea-74dbbef5052f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:19:07.035485 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f2a32a8-9bf8-4213-beea-74dbbef5052f None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:19:07.038226 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 369/744] 10.222.0.49 () {56 vars in 1148 bytes} [Wed Jun 15 10:19:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:19:07.039709 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c75583-04d7-4e61-b69c-ad9b490b0c22 None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:07.040152 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c75583-04d7-4e61-b69c-ad9b490b0c22 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:07.040427 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c75583-04d7-4e61-b69c-ad9b490b0c22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:07.053343 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application [None req-2f2a32a8-9bf8-4213-beea-74dbbef5052f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 10:19:07.053343 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:19:07.053343 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:19:07.053343 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:19:07.053343 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:19:07.053343 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:19:07.053343 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:19:07.053343 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:19:07.053343 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:19:07.053343 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:19:07.053343 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:19:07.053343 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:19:07.053343 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 10:19:07.053343 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 10:19:07.053343 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 10:19:07.054264 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 10:19:07.054264 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:19:07.054264 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:19:07.054264 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:19:07.054264 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:19:07.054264 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:19:07.054264 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:19:07.054264 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:19:07.054264 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application with Lock( Jun 15 10:19:07.054264 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:19:07.054264 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:19:07.054264 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:19:07.054264 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:19:07.054264 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:19:07.054264 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:19:07.054264 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:19:07.054264 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:19:07.055879 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 10:19:07.055879 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 10:19:07.055879 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 10:19:07.055879 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application Jun 15 10:19:07.055879 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 376/745] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Jun 15 10:19:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:19:07.055879 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2a32a8-9bf8-4213-beea-74dbbef5052f None admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:19:07.063160 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-62c75583-04d7-4e61-b69c-ad9b490b0c22 None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:19:07.063160 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-62c75583-04d7-4e61-b69c-ad9b490b0c22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:19:07.123688 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=100 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:19:07.132019 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 370/746] 10.222.0.49 () {58 vars in 1023 bytes} [Wed Jun 15 10:19:07 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 10:19:07.132838 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c75583-04d7-4e61-b69c-ad9b490b0c22 None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:19:07.135162 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:19:07.180067 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2a32a8-9bf8-4213-beea-74dbbef5052f None admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:19:07.217702 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117dcc68-7d0b-461e-9e21-d643cb599183 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:19:07.218460 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117dcc68-7d0b-461e-9e21-d643cb599183 None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:07.219055 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117dcc68-7d0b-461e-9e21-d643cb599183 None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:07.219055 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117dcc68-7d0b-461e-9e21-d643cb599183 None admin] PATH_INFO: `/v3/domains` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:07.219755 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-117dcc68-7d0b-461e-9e21-d643cb599183 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:19:07.220059 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-117dcc68-7d0b-461e-9e21-d643cb599183 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:19:07.221441 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-117dcc68-7d0b-461e-9e21-d643cb599183 None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:19:07.237614 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 377/747] 10.222.0.49 () {56 vars in 1109 bytes} [Wed Jun 15 10:19:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:19:07.237614 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117dcc68-7d0b-461e-9e21-d643cb599183 None admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:19:07.243361 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c75583-04d7-4e61-b69c-ad9b490b0c22 None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:19:07.278478 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e1ea8f-75d2-46f1-a489-a527ab46ac82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:19:07.279406 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e1ea8f-75d2-46f1-a489-a527ab46ac82 None admin] REQUEST_METHOD: `PUT` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:07.279690 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e1ea8f-75d2-46f1-a489-a527ab46ac82 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:07.279963 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e1ea8f-75d2-46f1-a489-a527ab46ac82 None admin] PATH_INFO: `/v3/projects/559b3e713f5844b6ae37151d56dd419f/users/5ac54528370d4747b6e8777fba9d0c82/roles/8051174e7a7947d5b991ebd793c2e543` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:07.281935 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e1ea8f-75d2-46f1-a489-a527ab46ac82 None admin] RBAC: Authorizing `identity:create_grant(project_id=559b3e713f5844b6ae37151d56dd419f, user_id=5ac54528370d4747b6e8777fba9d0c82, role_id=8051174e7a7947d5b991ebd793c2e543)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:19:07.283405 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e1ea8f-75d2-46f1-a489-a527ab46ac82 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:19:07.305292 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 371/748] 10.222.0.49 () {58 vars in 1439 bytes} [Wed Jun 15 10:19:07 2022] PUT /identity/v3/projects/559b3e713f5844b6ae37151d56dd419f/users/5ac54528370d4747b6e8777fba9d0c82/roles/8051174e7a7947d5b991ebd793c2e543 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:19:07.305292 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e1ea8f-75d2-46f1-a489-a527ab46ac82 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:19:07.361487 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117dcc68-7d0b-461e-9e21-d643cb599183 None admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:19:07.389154 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=100 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:19:07.402292 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:19:07.403573 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19aac52a-27fc-4844-a01d-0cb40d500fe6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:19:07.404438 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19aac52a-27fc-4844-a01d-0cb40d500fe6 None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:07.404819 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19aac52a-27fc-4844-a01d-0cb40d500fe6 None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:07.405212 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19aac52a-27fc-4844-a01d-0cb40d500fe6 None admin] PATH_INFO: `/v3/domains` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:07.406110 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19aac52a-27fc-4844-a01d-0cb40d500fe6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:19:07.406553 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19aac52a-27fc-4844-a01d-0cb40d500fe6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:19:07.408124 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19aac52a-27fc-4844-a01d-0cb40d500fe6 None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:19:07.422568 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 378/749] 10.222.0.49 () {56 vars in 1109 bytes} [Wed Jun 15 10:19:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:19:07.423725 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19aac52a-27fc-4844-a01d-0cb40d500fe6 None admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:19:07.459259 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e1ea8f-75d2-46f1-a489-a527ab46ac82 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:19:07.497792 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5087c32f-e47f-49dd-86cb-79d4b407be6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:19:07.498631 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5087c32f-e47f-49dd-86cb-79d4b407be6c None admin] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:07.499006 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5087c32f-e47f-49dd-86cb-79d4b407be6c None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:07.499407 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5087c32f-e47f-49dd-86cb-79d4b407be6c None admin] PATH_INFO: `/v3/services` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:07.500219 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5087c32f-e47f-49dd-86cb-79d4b407be6c None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:19:07.501729 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5087c32f-e47f-49dd-86cb-79d4b407be6c None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:19:07.519507 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 372/750] 10.222.0.49 () {60 vars in 1141 bytes} [Wed Jun 15 10:19:07 2022] POST /identity/v3/services => generated 240 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 10:19:07.521806 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5087c32f-e47f-49dd-86cb-79d4b407be6c None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:19:07.561490 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19aac52a-27fc-4844-a01d-0cb40d500fe6 None admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:19:07.615929 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17cabc9-8008-4aa3-8201-4705476e1fc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:19:07.617462 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17cabc9-8008-4aa3-8201-4705476e1fc8 None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:07.617876 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17cabc9-8008-4aa3-8201-4705476e1fc8 None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:07.618245 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17cabc9-8008-4aa3-8201-4705476e1fc8 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:07.628728 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c17cabc9-8008-4aa3-8201-4705476e1fc8 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:19:07.630477 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c17cabc9-8008-4aa3-8201-4705476e1fc8 None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:19:07.647332 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application [None req-c17cabc9-8008-4aa3-8201-4705476e1fc8 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 15 10:19:07.647332 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:19:07.647332 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:19:07.647332 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:19:07.647332 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:19:07.647332 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:19:07.647332 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:19:07.647332 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:19:07.647332 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:19:07.647332 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:19:07.647332 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:19:07.647332 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:19:07.647332 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 15 10:19:07.647332 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 15 10:19:07.647332 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 15 10:19:07.647332 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 15 10:19:07.648569 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:19:07.648569 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:19:07.648569 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 15 10:19:07.648569 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 10:19:07.648569 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 15 10:19:07.648569 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 10:19:07.648569 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:19:07.648569 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:19:07.648569 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:19:07.648569 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:19:07.648569 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:19:07.648569 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application with Lock( Jun 15 10:19:07.648569 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:19:07.648569 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:19:07.648569 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:19:07.648569 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:19:07.648569 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:19:07.648569 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:19:07.649780 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:19:07.649780 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:19:07.649780 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 15 10:19:07.649780 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 15 10:19:07.649780 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 15 10:19:07.649780 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 15 10:19:07.649780 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 15 10:19:07.649780 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 15 10:19:07.649780 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 15 10:19:07.649780 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application Jun 15 10:19:07.652273 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 379/751] 10.222.0.49 () {56 vars in 1117 bytes} [Wed Jun 15 10:19:07 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:19:07.657031 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5087c32f-e47f-49dd-86cb-79d4b407be6c None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:19:07.658225 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17cabc9-8008-4aa3-8201-4705476e1fc8 None admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:19:07.686188 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:19:07.705288 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb23f387-47c0-4967-ac1e-2c3792978252 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:19:07.706353 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb23f387-47c0-4967-ac1e-2c3792978252 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:07.706818 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb23f387-47c0-4967-ac1e-2c3792978252 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:07.707343 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb23f387-47c0-4967-ac1e-2c3792978252 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:07.711297 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=100 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:19:07.716353 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb23f387-47c0-4967-ac1e-2c3792978252 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:19:07.718085 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb23f387-47c0-4967-ac1e-2c3792978252 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:19:07.733836 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application [None req-eb23f387-47c0-4967-ac1e-2c3792978252 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 15 10:19:07.733836 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:19:07.733836 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:19:07.733836 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:19:07.733836 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:19:07.733836 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:19:07.733836 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:19:07.733836 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:19:07.733836 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:19:07.733836 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:19:07.733836 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:19:07.733836 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:19:07.733836 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 15 10:19:07.733836 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 15 10:19:07.733836 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 15 10:19:07.733836 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 15 10:19:07.735053 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:19:07.735053 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:19:07.735053 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 15 10:19:07.735053 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 10:19:07.735053 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 15 10:19:07.735053 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 10:19:07.735053 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:19:07.735053 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:19:07.735053 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:19:07.735053 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:19:07.735053 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:19:07.735053 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application with Lock( Jun 15 10:19:07.735053 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:19:07.735053 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:19:07.735053 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:19:07.735053 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:19:07.735053 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:19:07.735053 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:19:07.736280 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:19:07.736280 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:19:07.736280 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 15 10:19:07.736280 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 15 10:19:07.736280 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 15 10:19:07.736280 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 15 10:19:07.736280 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 15 10:19:07.736280 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 15 10:19:07.736280 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 15 10:19:07.736280 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Jun 15 10:19:07.738248 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 373/752] 10.222.0.49 () {56 vars in 1096 bytes} [Wed Jun 15 10:19:07 2022] GET /identity/v3/users/swift => generated 83 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:19:07.743780 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb23f387-47c0-4967-ac1e-2c3792978252 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:19:07.799149 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17cabc9-8008-4aa3-8201-4705476e1fc8 None admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:19:07.838652 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d498eb-2c67-416a-b575-7ea4ed131c82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:19:07.839680 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d498eb-2c67-416a-b575-7ea4ed131c82 None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:07.840110 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d498eb-2c67-416a-b575-7ea4ed131c82 None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:07.840532 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d498eb-2c67-416a-b575-7ea4ed131c82 None admin] PATH_INFO: `/v3/users` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:07.841537 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d498eb-2c67-416a-b575-7ea4ed131c82 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:19:07.841993 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d498eb-2c67-416a-b575-7ea4ed131c82 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:19:07.843604 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d498eb-2c67-416a-b575-7ea4ed131c82 None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:19:07.867010 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 380/753] 10.222.0.49 () {56 vars in 1149 bytes} [Wed Jun 15 10:19:07 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:19:07.867530 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb23f387-47c0-4967-ac1e-2c3792978252 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:19:07.868322 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d498eb-2c67-416a-b575-7ea4ed131c82 None admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:19:07.913305 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a697c50-d038-4721-8d1f-d914b5dda36d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:19:07.914218 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a697c50-d038-4721-8d1f-d914b5dda36d None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:07.914718 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a697c50-d038-4721-8d1f-d914b5dda36d None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:07.915243 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a697c50-d038-4721-8d1f-d914b5dda36d None admin] PATH_INFO: `/v3/users` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:07.916314 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a697c50-d038-4721-8d1f-d914b5dda36d None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:19:07.918167 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a697c50-d038-4721-8d1f-d914b5dda36d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:19:07.919852 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a697c50-d038-4721-8d1f-d914b5dda36d None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:19:07.942983 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 374/754] 10.222.0.49 () {56 vars in 1135 bytes} [Wed Jun 15 10:19:07 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 10:19:07.948791 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a697c50-d038-4721-8d1f-d914b5dda36d None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:19:07.983207 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d498eb-2c67-416a-b575-7ea4ed131c82 None admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:19:08.005831 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:19:08.015481 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=100 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:19:08.026626 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07076f87-cb17-4c4f-8561-790cf07be50e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:19:08.027790 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07076f87-cb17-4c4f-8561-790cf07be50e None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:08.028668 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07076f87-cb17-4c4f-8561-790cf07be50e None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:08.029402 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07076f87-cb17-4c4f-8561-790cf07be50e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:08.083077 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a697c50-d038-4721-8d1f-d914b5dda36d None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:19:08.120315 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca54c504-c133-4896-aa4b-2705a5d870e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:19:08.120978 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca54c504-c133-4896-aa4b-2705a5d870e2 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:08.121261 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca54c504-c133-4896-aa4b-2705a5d870e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:08.121535 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca54c504-c133-4896-aa4b-2705a5d870e2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:08.132440 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07076f87-cb17-4c4f-8561-790cf07be50e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:19:08.133908 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07076f87-cb17-4c4f-8561-790cf07be50e None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:19:08.202013 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca54c504-c133-4896-aa4b-2705a5d870e2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:19:08.203520 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca54c504-c133-4896-aa4b-2705a5d870e2 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:19:08.247463 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:19:08.273845 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=100 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:19:08.285995 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 381/755] 10.222.0.49 () {58 vars in 1282 bytes} [Wed Jun 15 10:19:07 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 418 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 15 10:19:08.292219 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07076f87-cb17-4c4f-8561-790cf07be50e None admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:19:08.310499 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 375/756] 10.222.0.49 () {58 vars in 1282 bytes} [Wed Jun 15 10:19:07 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 15 10:19:08.316079 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca54c504-c133-4896-aa4b-2705a5d870e2 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:19:08.418195 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca54c504-c133-4896-aa4b-2705a5d870e2 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:19:08.468171 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6a9669-b8d3-4c14-b5ef-6f3bbc5b02b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:19:08.469762 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6a9669-b8d3-4c14-b5ef-6f3bbc5b02b9 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:08.470045 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6a9669-b8d3-4c14-b5ef-6f3bbc5b02b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:08.470317 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6a9669-b8d3-4c14-b5ef-6f3bbc5b02b9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:08.489939 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c6a9669-b8d3-4c14-b5ef-6f3bbc5b02b9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:19:08.494218 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c6a9669-b8d3-4c14-b5ef-6f3bbc5b02b9 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:19:08.495978 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:19:08.499346 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07076f87-cb17-4c4f-8561-790cf07be50e None admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:19:08.526696 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application [None req-1c6a9669-b8d3-4c14-b5ef-6f3bbc5b02b9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 10:19:08.526696 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:19:08.526696 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:19:08.526696 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:19:08.526696 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:19:08.526696 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:19:08.526696 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:19:08.526696 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:19:08.526696 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:19:08.526696 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:19:08.526696 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:19:08.526696 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:19:08.526696 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 10:19:08.526696 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 10:19:08.526696 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 10:19:08.527808 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 10:19:08.527808 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:19:08.527808 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:19:08.527808 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:19:08.527808 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:19:08.527808 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:19:08.527808 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:19:08.527808 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:19:08.527808 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application with Lock( Jun 15 10:19:08.527808 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:19:08.527808 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:19:08.527808 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:19:08.527808 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:19:08.527808 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:19:08.527808 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:19:08.527808 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:19:08.527808 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:19:08.528814 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 10:19:08.528814 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 10:19:08.528814 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 10:19:08.528814 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Jun 15 10:19:08.535528 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 376/757] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Jun 15 10:19:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:19:08.536771 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6a9669-b8d3-4c14-b5ef-6f3bbc5b02b9 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:19:08.566351 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5a76da-0fa9-497f-acc6-887c226021cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:19:08.567168 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5a76da-0fa9-497f-acc6-887c226021cb None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:08.567455 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5a76da-0fa9-497f-acc6-887c226021cb None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:08.567729 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5a76da-0fa9-497f-acc6-887c226021cb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:08.590116 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f5a76da-0fa9-497f-acc6-887c226021cb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:19:08.591612 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f5a76da-0fa9-497f-acc6-887c226021cb None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:19:08.631322 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application [None req-3f5a76da-0fa9-497f-acc6-887c226021cb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 10:19:08.631322 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:19:08.631322 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:19:08.631322 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:19:08.631322 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:19:08.631322 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:19:08.631322 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:19:08.631322 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:19:08.631322 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:19:08.631322 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:19:08.631322 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:19:08.631322 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:19:08.631322 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 10:19:08.631322 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 10:19:08.631322 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 10:19:08.632302 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 10:19:08.632302 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:19:08.632302 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:19:08.632302 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:19:08.632302 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:19:08.632302 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:19:08.632302 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:19:08.632302 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:19:08.632302 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application with Lock( Jun 15 10:19:08.632302 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:19:08.632302 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:19:08.632302 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:19:08.632302 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:19:08.632302 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:19:08.632302 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:19:08.632302 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:19:08.632302 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:19:08.633222 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 10:19:08.633222 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 10:19:08.633222 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 10:19:08.633222 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application Jun 15 10:19:08.633222 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 382/758] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Jun 15 10:19:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 341 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 10:19:08.639664 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5a76da-0fa9-497f-acc6-887c226021cb None admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:19:08.702415 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6a9669-b8d3-4c14-b5ef-6f3bbc5b02b9 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:19:08.741374 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fbdd3cd-ddfb-4808-ab46-5e77c446f2eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:19:08.741965 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbdd3cd-ddfb-4808-ab46-5e77c446f2eb None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:08.742247 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbdd3cd-ddfb-4808-ab46-5e77c446f2eb None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:08.742521 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbdd3cd-ddfb-4808-ab46-5e77c446f2eb None admin] PATH_INFO: `/v3/domains` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:08.743266 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fbdd3cd-ddfb-4808-ab46-5e77c446f2eb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:19:08.743570 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fbdd3cd-ddfb-4808-ab46-5e77c446f2eb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:19:08.745094 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fbdd3cd-ddfb-4808-ab46-5e77c446f2eb None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:19:08.753815 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=100 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:19:08.758509 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 377/759] 10.222.0.49 () {56 vars in 1109 bytes} [Wed Jun 15 10:19:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 10:19:08.764483 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fbdd3cd-ddfb-4808-ab46-5e77c446f2eb None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:19:08.853418 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:19:08.876123 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5a76da-0fa9-497f-acc6-887c226021cb None admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:19:08.928168 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9d6437-7454-4632-8950-ace6ddbb4839 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:19:08.929207 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9d6437-7454-4632-8950-ace6ddbb4839 None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:08.929784 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9d6437-7454-4632-8950-ace6ddbb4839 None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:08.930313 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9d6437-7454-4632-8950-ace6ddbb4839 None admin] PATH_INFO: `/v3/domains` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:08.931278 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc9d6437-7454-4632-8950-ace6ddbb4839 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:19:08.931789 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc9d6437-7454-4632-8950-ace6ddbb4839 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:19:08.933898 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc9d6437-7454-4632-8950-ace6ddbb4839 None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:19:08.946816 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 383/760] 10.222.0.49 () {56 vars in 1109 bytes} [Wed Jun 15 10:19:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:19:08.953304 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9d6437-7454-4632-8950-ace6ddbb4839 None admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:19:08.957660 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fbdd3cd-ddfb-4808-ab46-5e77c446f2eb None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:19:09.021173 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0220a0f-1591-44c9-bea7-da7a7cd55724 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:19:09.021906 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0220a0f-1591-44c9-bea7-da7a7cd55724 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:09.022194 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0220a0f-1591-44c9-bea7-da7a7cd55724 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:09.022474 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0220a0f-1591-44c9-bea7-da7a7cd55724 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:09.032324 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0220a0f-1591-44c9-bea7-da7a7cd55724 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:19:09.037434 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0220a0f-1591-44c9-bea7-da7a7cd55724 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:19:09.059167 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application [None req-b0220a0f-1591-44c9-bea7-da7a7cd55724 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 10:19:09.059167 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:19:09.059167 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:19:09.059167 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:19:09.059167 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:19:09.059167 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:19:09.059167 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:19:09.059167 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:19:09.059167 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:19:09.059167 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:19:09.059167 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:19:09.059167 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:19:09.059167 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 15 10:19:09.059167 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 15 10:19:09.059167 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 15 10:19:09.060102 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 15 10:19:09.060102 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:19:09.060102 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:19:09.060102 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:19:09.060102 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:19:09.060102 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:19:09.060102 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:19:09.060102 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:19:09.060102 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application with Lock( Jun 15 10:19:09.060102 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:19:09.060102 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:19:09.060102 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:19:09.060102 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:19:09.060102 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:19:09.060102 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:19:09.060102 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:19:09.060102 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:19:09.061003 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 15 10:19:09.061003 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 15 10:19:09.061003 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 15 10:19:09.061003 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 15 10:19:09.061003 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 15 10:19:09.061003 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 15 10:19:09.061003 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 10:19:09.061003 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Jun 15 10:19:09.063699 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 378/761] 10.222.0.49 () {56 vars in 1111 bytes} [Wed Jun 15 10:19:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 300 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 10:19:09.075130 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0220a0f-1591-44c9-bea7-da7a7cd55724 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:19:09.088909 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9d6437-7454-4632-8950-ace6ddbb4839 None admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:19:09.103395 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=100 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:19:09.161348 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61544ab8-af14-4216-b1e0-510b8cf3616e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:19:09.162163 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61544ab8-af14-4216-b1e0-510b8cf3616e None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:09.162442 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61544ab8-af14-4216-b1e0-510b8cf3616e None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:09.162714 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61544ab8-af14-4216-b1e0-510b8cf3616e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:09.178907 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61544ab8-af14-4216-b1e0-510b8cf3616e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:19:09.181225 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61544ab8-af14-4216-b1e0-510b8cf3616e None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:19:09.203135 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application [None req-61544ab8-af14-4216-b1e0-510b8cf3616e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 10:19:09.203135 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:19:09.203135 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:19:09.203135 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:19:09.203135 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:19:09.203135 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:19:09.203135 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:19:09.203135 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:19:09.203135 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:19:09.203135 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:19:09.203135 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:19:09.203135 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:19:09.203135 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 15 10:19:09.203135 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 15 10:19:09.203135 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 15 10:19:09.204292 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 15 10:19:09.204292 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:19:09.204292 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:19:09.204292 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:19:09.204292 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:19:09.204292 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:19:09.204292 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:19:09.204292 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:19:09.204292 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application with Lock( Jun 15 10:19:09.204292 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:19:09.204292 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:19:09.204292 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:19:09.204292 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:19:09.204292 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:19:09.204292 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:19:09.204292 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:19:09.204292 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:19:09.205420 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 15 10:19:09.205420 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 15 10:19:09.205420 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 15 10:19:09.205420 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 15 10:19:09.205420 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 15 10:19:09.205420 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 15 10:19:09.205420 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 10:19:09.205420 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application Jun 15 10:19:09.207576 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 384/762] 10.222.0.49 () {56 vars in 1111 bytes} [Wed Jun 15 10:19:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:19:09.213388 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf399ec-bece-4153-8dec-afae9febaf7f None None] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:09.213953 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf399ec-bece-4153-8dec-afae9febaf7f None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:09.214424 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf399ec-bece-4153-8dec-afae9febaf7f None None] PATH_INFO: `/` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:09.216249 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 385/763] 10.222.0.49 () {54 vars in 926 bytes} [Wed Jun 15 10:19:09 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:19:09.217177 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf399ec-bece-4153-8dec-afae9febaf7f None None] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:19:09.256037 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0220a0f-1591-44c9-bea7-da7a7cd55724 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:19:09.274359 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:19:09.305759 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a3e94b-61d8-44d8-923f-44e797ff4cc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:19:09.306798 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a3e94b-61d8-44d8-923f-44e797ff4cc7 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:09.307865 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a3e94b-61d8-44d8-923f-44e797ff4cc7 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:09.308345 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a3e94b-61d8-44d8-923f-44e797ff4cc7 None admin] PATH_INFO: `/v3/projects` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:09.309862 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a3e94b-61d8-44d8-923f-44e797ff4cc7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:19:09.310392 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a3e94b-61d8-44d8-923f-44e797ff4cc7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:19:09.312506 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a3e94b-61d8-44d8-923f-44e797ff4cc7 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:19:09.329302 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 379/764] 10.222.0.49 () {56 vars in 1148 bytes} [Wed Jun 15 10:19:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:19:09.331717 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e898f17-cf26-4b06-b313-3fba9b0253f2 None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:09.332459 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e898f17-cf26-4b06-b313-3fba9b0253f2 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:09.333381 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e898f17-cf26-4b06-b313-3fba9b0253f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:09.337151 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf399ec-bece-4153-8dec-afae9febaf7f None None] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:19:09.354625 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-6e898f17-cf26-4b06-b313-3fba9b0253f2 None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:19:09.356814 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-6e898f17-cf26-4b06-b313-3fba9b0253f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:19:09.375931 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7b536b-0134-457b-be0f-c1f4c9d46f3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:19:09.376321 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7b536b-0134-457b-be0f-c1f4c9d46f3d None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:09.376668 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7b536b-0134-457b-be0f-c1f4c9d46f3d None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:09.376952 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7b536b-0134-457b-be0f-c1f4c9d46f3d None admin] PATH_INFO: `/v3/projects` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:09.377792 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7b536b-0134-457b-be0f-c1f4c9d46f3d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:19:09.378150 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7b536b-0134-457b-be0f-c1f4c9d46f3d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:19:09.379719 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7b536b-0134-457b-be0f-c1f4c9d46f3d None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:19:09.396917 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 386/765] 10.222.0.49 () {56 vars in 1148 bytes} [Wed Jun 15 10:19:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:19:09.396917 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7b536b-0134-457b-be0f-c1f4c9d46f3d None admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:19:09.428528 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=100 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:19:09.454834 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 380/766] 10.222.0.49 () {58 vars in 1023 bytes} [Wed Jun 15 10:19:09 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 10:19:09.456033 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e898f17-cf26-4b06-b313-3fba9b0253f2 None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:19:09.528650 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7b536b-0134-457b-be0f-c1f4c9d46f3d None admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:19:09.564407 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4e49fa-2974-4a62-9a84-7678fb0b8359 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:19:09.565069 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4e49fa-2974-4a62-9a84-7678fb0b8359 None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:09.565407 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4e49fa-2974-4a62-9a84-7678fb0b8359 None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:09.565684 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4e49fa-2974-4a62-9a84-7678fb0b8359 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:09.566523 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e4e49fa-2974-4a62-9a84-7678fb0b8359 None admin] RBAC: Adding query filter params (role.id=8051174e7a7947d5b991ebd793c2e543, scope.project.id=559b3e713f5844b6ae37151d56dd419f, user.id=3a69f99210444f9696e8b22615fb4a41) {{(pid=80119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:19:09.567060 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e4e49fa-2974-4a62-9a84-7678fb0b8359 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:19:09.568207 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e898f17-cf26-4b06-b313-3fba9b0253f2 None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:19:09.568586 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e4e49fa-2974-4a62-9a84-7678fb0b8359 None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:19:09.570443 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:19:09.574765 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 387/767] 10.222.0.49 () {56 vars in 1374 bytes} [Wed Jun 15 10:19:09 2022] GET /identity/v3/role_assignments?user.id=3a69f99210444f9696e8b22615fb4a41&scope.project.id=559b3e713f5844b6ae37151d56dd419f&role.id=8051174e7a7947d5b991ebd793c2e543 => generated 259 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:19:09.575769 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4e49fa-2974-4a62-9a84-7678fb0b8359 None admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:19:09.611785 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b95bad-face-4256-bc01-6d03637e6ef3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:19:09.612849 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b95bad-face-4256-bc01-6d03637e6ef3 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:09.613361 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b95bad-face-4256-bc01-6d03637e6ef3 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:09.613865 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b95bad-face-4256-bc01-6d03637e6ef3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:09.615126 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b95bad-face-4256-bc01-6d03637e6ef3 None admin] RBAC: Adding query filter params (role.id=16e76452bb5d4ce68a119ad3c9bde41c, scope.project.id=559b3e713f5844b6ae37151d56dd419f, user.id=103e10244eca444b86b9c51ff314d445) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:19:09.615675 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b95bad-face-4256-bc01-6d03637e6ef3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:19:09.617474 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b95bad-face-4256-bc01-6d03637e6ef3 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:19:09.624886 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 381/768] 10.222.0.49 () {56 vars in 1374 bytes} [Wed Jun 15 10:19:09 2022] GET /identity/v3/role_assignments?user.id=103e10244eca444b86b9c51ff314d445&scope.project.id=559b3e713f5844b6ae37151d56dd419f&role.id=16e76452bb5d4ce68a119ad3c9bde41c => generated 259 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:19:09.683753 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=100 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:19:09.693882 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4e49fa-2974-4a62-9a84-7678fb0b8359 None admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:19:09.736224 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c70727-a684-4e19-8b7e-e9b8f45872db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:19:09.737055 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c70727-a684-4e19-8b7e-e9b8f45872db None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:09.737344 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c70727-a684-4e19-8b7e-e9b8f45872db None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:09.737627 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c70727-a684-4e19-8b7e-e9b8f45872db None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:09.738292 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c70727-a684-4e19-8b7e-e9b8f45872db None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:19:09.739726 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c70727-a684-4e19-8b7e-e9b8f45872db None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:19:09.753490 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application [None req-87c70727-a684-4e19-8b7e-e9b8f45872db None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 15 10:19:09.753490 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:19:09.753490 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 15 10:19:09.753490 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 15 10:19:09.753490 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 15 10:19:09.753490 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 15 10:19:09.753490 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 15 10:19:09.753490 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 10:19:09.753490 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 15 10:19:09.753490 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application Jun 15 10:19:09.753490 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 15 10:19:09.753490 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application Jun 15 10:19:09.753490 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:19:09.753490 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:19:09.753490 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:19:09.753490 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:19:09.754517 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:19:09.754517 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:19:09.754517 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:19:09.754517 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:19:09.754517 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:19:09.754517 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:19:09.754517 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:19:09.754517 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 15 10:19:09.754517 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 15 10:19:09.754517 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 15 10:19:09.754517 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 15 10:19:09.754517 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:19:09.754517 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:19:09.754517 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:19:09.754517 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:19:09.754517 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:19:09.755405 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:19:09.755405 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:19:09.755405 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application with Lock( Jun 15 10:19:09.755405 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:19:09.755405 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:19:09.755405 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:19:09.755405 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:19:09.755405 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:19:09.755405 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:19:09.755405 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:19:09.755405 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:19:09.755405 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 15 10:19:09.755405 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 10:19:09.755405 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 15 10:19:09.755405 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application Jun 15 10:19:09.759238 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 388/769] 10.222.0.49 () {56 vars in 1129 bytes} [Wed Jun 15 10:19:09 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:19:09.764330 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b95bad-face-4256-bc01-6d03637e6ef3 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:19:09.894450 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b95bad-face-4256-bc01-6d03637e6ef3 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:19:09.932312 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e78cea-d282-4a97-846c-9da7dd9e3ef1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:19:09.933385 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e78cea-d282-4a97-846c-9da7dd9e3ef1 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:09.933878 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e78cea-d282-4a97-846c-9da7dd9e3ef1 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:09.934358 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e78cea-d282-4a97-846c-9da7dd9e3ef1 None admin] PATH_INFO: `/v3/services` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:09.935381 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e78cea-d282-4a97-846c-9da7dd9e3ef1 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:19:09.935912 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e78cea-d282-4a97-846c-9da7dd9e3ef1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:19:09.937580 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e78cea-d282-4a97-846c-9da7dd9e3ef1 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:19:09.943721 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 382/770] 10.222.0.49 () {56 vars in 1124 bytes} [Wed Jun 15 10:19:09 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:19:09.951126 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c70727-a684-4e19-8b7e-e9b8f45872db None admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:19:10.008196 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684f52e4-6243-485d-a6b9-364d313db4a8 None None] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:10.008784 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684f52e4-6243-485d-a6b9-364d313db4a8 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:10.009097 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684f52e4-6243-485d-a6b9-364d313db4a8 None None] PATH_INFO: `/` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:10.011137 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 383/771] 10.222.0.49 () {54 vars in 926 bytes} [Wed Jun 15 10:19:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 15 10:19:10.017862 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23402eb-482f-4481-8035-ea779c585b8d None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:10.018160 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23402eb-482f-4481-8035-ea779c585b8d None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:10.018437 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23402eb-482f-4481-8035-ea779c585b8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:10.053203 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-f23402eb-482f-4481-8035-ea779c585b8d None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:19:10.054332 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-f23402eb-482f-4481-8035-ea779c585b8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:19:10.107456 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:19:10.119367 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c70727-a684-4e19-8b7e-e9b8f45872db None admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:19:10.190573 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1788c75f-6f3f-4042-9b53-0e6fd1d6cef7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:19:10.190978 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 384/772] 10.222.0.49 () {58 vars in 1023 bytes} [Wed Jun 15 10:19:10 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 10:19:10.191934 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1788c75f-6f3f-4042-9b53-0e6fd1d6cef7 None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:10.192450 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1788c75f-6f3f-4042-9b53-0e6fd1d6cef7 None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:10.192995 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1788c75f-6f3f-4042-9b53-0e6fd1d6cef7 None admin] PATH_INFO: `/v3/services` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:10.193959 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1788c75f-6f3f-4042-9b53-0e6fd1d6cef7 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:19:10.194516 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1788c75f-6f3f-4042-9b53-0e6fd1d6cef7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:19:10.196239 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1788c75f-6f3f-4042-9b53-0e6fd1d6cef7 None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:19:10.203052 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 389/773] 10.222.0.49 () {56 vars in 1124 bytes} [Wed Jun 15 10:19:09 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:19:10.212228 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23402eb-482f-4481-8035-ea779c585b8d None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:19:10.246815 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1788c75f-6f3f-4042-9b53-0e6fd1d6cef7 None admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:19:10.271279 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=100 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:19:10.331018 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23402eb-482f-4481-8035-ea779c585b8d None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:19:10.369848 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87998505-b917-4c89-995e-c062edafa605 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:19:10.370520 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87998505-b917-4c89-995e-c062edafa605 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:10.370812 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87998505-b917-4c89-995e-c062edafa605 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:10.371109 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87998505-b917-4c89-995e-c062edafa605 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:10.371905 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87998505-b917-4c89-995e-c062edafa605 None admin] RBAC: Adding query filter params (interface=public, service_id=719fa543aa8d47b69c79267676af64ab, region_id=RegionOne) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:19:10.372218 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87998505-b917-4c89-995e-c062edafa605 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:19:10.373592 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87998505-b917-4c89-995e-c062edafa605 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:19:10.382335 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 385/774] 10.222.0.49 () {56 vars in 1251 bytes} [Wed Jun 15 10:19:10 2022] GET /identity/v3/endpoints?service_id=719fa543aa8d47b69c79267676af64ab&interface=public®ion_id=RegionOne => generated 194 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:19:10.383721 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645ac7a2-217a-4e2f-b218-0c530dcb8909 None None] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:10.384011 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645ac7a2-217a-4e2f-b218-0c530dcb8909 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:10.384290 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645ac7a2-217a-4e2f-b218-0c530dcb8909 None None] PATH_INFO: `/` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:10.389645 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 386/775] 10.222.0.49 () {54 vars in 926 bytes} [Wed Jun 15 10:19:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:19:10.389645 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645ac7a2-217a-4e2f-b218-0c530dcb8909 None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:19:10.395671 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1788c75f-6f3f-4042-9b53-0e6fd1d6cef7 None admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:19:10.414837 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:19:10.440674 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc80d1b3-928c-4065-ac54-9879ddabcaf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:19:10.441507 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc80d1b3-928c-4065-ac54-9879ddabcaf4 None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:10.441852 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc80d1b3-928c-4065-ac54-9879ddabcaf4 None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:10.442134 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc80d1b3-928c-4065-ac54-9879ddabcaf4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:10.453157 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc80d1b3-928c-4065-ac54-9879ddabcaf4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 10:19:10.461115 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application [None req-bc80d1b3-928c-4065-ac54-9879ddabcaf4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 10:19:10.461115 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:19:10.461115 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:19:10.461115 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:19:10.461115 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:19:10.461115 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:19:10.461115 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:19:10.461115 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:19:10.461115 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:19:10.461115 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:19:10.461115 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:19:10.461115 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:19:10.461115 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 10:19:10.461115 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 10:19:10.461115 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 10:19:10.461115 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 10:19:10.462083 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 10:19:10.462083 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 10:19:10.462083 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 10:19:10.462083 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 10:19:10.462083 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:19:10.462083 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:19:10.462083 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:19:10.462083 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:19:10.462083 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:19:10.462083 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:19:10.462083 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:19:10.462083 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application with Lock( Jun 15 10:19:10.462083 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:19:10.462083 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:19:10.462083 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:19:10.462083 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:19:10.462930 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:19:10.462930 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:19:10.462930 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:19:10.462930 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:19:10.462930 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 10:19:10.462930 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 10:19:10.462930 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 10:19:10.462930 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 10:19:10.462930 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 10:19:10.462930 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 10:19:10.462930 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 10:19:10.462930 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application Jun 15 10:19:10.462930 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 390/776] 10.222.0.49 () {56 vars in 1096 bytes} [Wed Jun 15 10:19:10 2022] GET /identity/v3/roles/admin => generated 83 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:19:10.464704 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40787cce-abcf-4c93-8df0-90b4dadd9cfa None None] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:10.464804 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40787cce-abcf-4c93-8df0-90b4dadd9cfa None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:10.465290 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40787cce-abcf-4c93-8df0-90b4dadd9cfa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:10.482732 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.auth.core [None req-40787cce-abcf-4c93-8df0-90b4dadd9cfa None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:19:10.484010 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-40787cce-abcf-4c93-8df0-90b4dadd9cfa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:19:10.520724 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645ac7a2-217a-4e2f-b218-0c530dcb8909 None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:19:10.571112 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 391/777] 10.222.0.49 () {58 vars in 1023 bytes} [Wed Jun 15 10:19:10 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 10:19:10.571112 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40787cce-abcf-4c93-8df0-90b4dadd9cfa None None] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:19:10.572012 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57542830-7a2e-4c7a-bd9a-7d062b5a36f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:19:10.574859 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57542830-7a2e-4c7a-bd9a-7d062b5a36f0 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:10.574859 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57542830-7a2e-4c7a-bd9a-7d062b5a36f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:10.574859 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57542830-7a2e-4c7a-bd9a-7d062b5a36f0 None admin] PATH_INFO: `/v3/services` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:10.574859 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57542830-7a2e-4c7a-bd9a-7d062b5a36f0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:19:10.575316 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57542830-7a2e-4c7a-bd9a-7d062b5a36f0 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:19:10.590602 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 387/778] 10.222.0.49 () {56 vars in 1087 bytes} [Wed Jun 15 10:19:10 2022] GET /identity/v3/services => generated 772 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:19:10.594246 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=100 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:19:10.596450 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57542830-7a2e-4c7a-bd9a-7d062b5a36f0 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:19:10.710270 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40787cce-abcf-4c93-8df0-90b4dadd9cfa None None] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:19:10.751333 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1711aa01-144b-4a5f-b3b6-c11d3e25ed8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:19:10.752530 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1711aa01-144b-4a5f-b3b6-c11d3e25ed8e None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:10.752822 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1711aa01-144b-4a5f-b3b6-c11d3e25ed8e None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:10.753253 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1711aa01-144b-4a5f-b3b6-c11d3e25ed8e None admin] PATH_INFO: `/v3/roles` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:10.754034 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1711aa01-144b-4a5f-b3b6-c11d3e25ed8e None admin] RBAC: Adding query filter params (name=admin) {{(pid=80119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:19:10.754349 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1711aa01-144b-4a5f-b3b6-c11d3e25ed8e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:19:10.755980 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1711aa01-144b-4a5f-b3b6-c11d3e25ed8e None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:19:10.767604 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 392/779] 10.222.0.49 () {56 vars in 1099 bytes} [Wed Jun 15 10:19:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:19:10.775357 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1711aa01-144b-4a5f-b3b6-c11d3e25ed8e None admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:19:10.815521 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:19:10.829719 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57542830-7a2e-4c7a-bd9a-7d062b5a36f0 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:19:10.868278 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac83fad4-ce3c-487c-8ebc-e55321d34342 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:19:10.869135 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac83fad4-ce3c-487c-8ebc-e55321d34342 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:10.869450 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac83fad4-ce3c-487c-8ebc-e55321d34342 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:10.869730 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac83fad4-ce3c-487c-8ebc-e55321d34342 None admin] PATH_INFO: `/v3/services/network` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:10.870387 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac83fad4-ce3c-487c-8ebc-e55321d34342 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:19:10.871928 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac83fad4-ce3c-487c-8ebc-e55321d34342 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:19:10.872906 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=100 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:19:10.887314 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application [None req-ac83fad4-ce3c-487c-8ebc-e55321d34342 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jun 15 10:19:10.887314 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:19:10.887314 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 15 10:19:10.887314 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 15 10:19:10.887314 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 15 10:19:10.887314 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 15 10:19:10.887314 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 15 10:19:10.887314 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 10:19:10.887314 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 15 10:19:10.887314 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Jun 15 10:19:10.887314 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 15 10:19:10.887314 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Jun 15 10:19:10.887314 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:19:10.887314 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:19:10.887314 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:19:10.887314 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:19:10.887314 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:19:10.888397 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:19:10.888397 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:19:10.888397 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:19:10.888397 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:19:10.888397 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:19:10.888397 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:19:10.888397 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 15 10:19:10.888397 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 15 10:19:10.888397 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 15 10:19:10.888397 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 15 10:19:10.888397 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:19:10.888397 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:19:10.888397 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:19:10.888397 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:19:10.888397 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:19:10.888397 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:19:10.889249 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:19:10.889249 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application with Lock( Jun 15 10:19:10.889249 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:19:10.889249 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:19:10.889249 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:19:10.889249 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:19:10.889249 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:19:10.889249 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:19:10.889249 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:19:10.889249 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:19:10.889249 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 15 10:19:10.889249 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 10:19:10.889249 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 15 10:19:10.889249 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Jun 15 10:19:10.892815 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 388/780] 10.222.0.49 () {56 vars in 1111 bytes} [Wed Jun 15 10:19:10 2022] GET /identity/v3/services/network => generated 88 bytes in 297 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:19:10.905206 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac83fad4-ce3c-487c-8ebc-e55321d34342 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:19:10.922473 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1711aa01-144b-4a5f-b3b6-c11d3e25ed8e None admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:19:10.994729 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713488a2-6c5d-45cc-9295-581b08c39aa0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:19:10.995631 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713488a2-6c5d-45cc-9295-581b08c39aa0 None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:10.995924 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713488a2-6c5d-45cc-9295-581b08c39aa0 None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:10.996261 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713488a2-6c5d-45cc-9295-581b08c39aa0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:11.025109 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-713488a2-6c5d-45cc-9295-581b08c39aa0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:19:11.027288 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-713488a2-6c5d-45cc-9295-581b08c39aa0 None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:19:11.058388 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application [None req-713488a2-6c5d-45cc-9295-581b08c39aa0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 10:19:11.058388 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:19:11.058388 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:19:11.058388 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:19:11.058388 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:19:11.058388 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:19:11.058388 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:19:11.058388 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:19:11.058388 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:19:11.058388 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:19:11.058388 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:19:11.058388 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:19:11.058388 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 10:19:11.058388 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 10:19:11.058388 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 10:19:11.059393 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 10:19:11.059393 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:19:11.059393 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:19:11.059393 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:19:11.059393 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:19:11.059393 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:19:11.059393 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:19:11.059393 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:19:11.059393 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application with Lock( Jun 15 10:19:11.059393 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:19:11.059393 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:19:11.059393 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:19:11.059393 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:19:11.059393 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:19:11.059393 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:19:11.059393 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:19:11.059393 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:19:11.060313 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 10:19:11.060313 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 10:19:11.060313 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 10:19:11.060313 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application Jun 15 10:19:11.060313 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 393/781] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Jun 15 10:19:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 286 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:19:11.073531 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713488a2-6c5d-45cc-9295-581b08c39aa0 None admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:19:11.095680 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac83fad4-ce3c-487c-8ebc-e55321d34342 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:19:11.161894 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd026d6-ee8b-4ea0-844a-2a8ddeaa7cb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:19:11.162488 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd026d6-ee8b-4ea0-844a-2a8ddeaa7cb3 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:11.162774 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd026d6-ee8b-4ea0-844a-2a8ddeaa7cb3 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:11.163053 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd026d6-ee8b-4ea0-844a-2a8ddeaa7cb3 None admin] PATH_INFO: `/v3/services` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:11.163798 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd026d6-ee8b-4ea0-844a-2a8ddeaa7cb3 None admin] RBAC: Adding query filter params (name=network) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:19:11.164165 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd026d6-ee8b-4ea0-844a-2a8ddeaa7cb3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:19:11.165984 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd026d6-ee8b-4ea0-844a-2a8ddeaa7cb3 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:19:11.171154 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 389/782] 10.222.0.49 () {56 vars in 1112 bytes} [Wed Jun 15 10:19:10 2022] GET /identity/v3/services?name=network => generated 124 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 10:19:11.177105 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd026d6-ee8b-4ea0-844a-2a8ddeaa7cb3 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:19:11.209850 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:19:11.284895 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713488a2-6c5d-45cc-9295-581b08c39aa0 None admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:19:11.328506 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=100 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:19:11.346910 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475fa96a-ae6f-44a7-b154-48b8d4105564 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:19:11.347611 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475fa96a-ae6f-44a7-b154-48b8d4105564 None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:11.347896 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475fa96a-ae6f-44a7-b154-48b8d4105564 None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:11.348176 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475fa96a-ae6f-44a7-b154-48b8d4105564 None admin] PATH_INFO: `/v3/domains` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:11.348903 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-475fa96a-ae6f-44a7-b154-48b8d4105564 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:19:11.349210 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-475fa96a-ae6f-44a7-b154-48b8d4105564 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:19:11.350597 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-475fa96a-ae6f-44a7-b154-48b8d4105564 None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:19:11.366133 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd026d6-ee8b-4ea0-844a-2a8ddeaa7cb3 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:19:11.377857 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 394/783] 10.222.0.49 () {56 vars in 1109 bytes} [Wed Jun 15 10:19:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:19:11.383227 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475fa96a-ae6f-44a7-b154-48b8d4105564 None admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:19:11.428605 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58bdf926-1688-4e18-ac91-2971efb1c695 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:19:11.429203 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bdf926-1688-4e18-ac91-2971efb1c695 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:11.429488 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bdf926-1688-4e18-ac91-2971efb1c695 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:11.429764 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bdf926-1688-4e18-ac91-2971efb1c695 None admin] PATH_INFO: `/v3/services` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:11.430474 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58bdf926-1688-4e18-ac91-2971efb1c695 None admin] RBAC: Adding query filter params (type=network) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:19:11.430779 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58bdf926-1688-4e18-ac91-2971efb1c695 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:19:11.432749 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58bdf926-1688-4e18-ac91-2971efb1c695 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:19:11.445060 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 390/784] 10.222.0.49 () {56 vars in 1112 bytes} [Wed Jun 15 10:19:11 2022] GET /identity/v3/services?type=network => generated 350 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 10:19:11.451510 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58bdf926-1688-4e18-ac91-2971efb1c695 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:19:11.571150 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475fa96a-ae6f-44a7-b154-48b8d4105564 None admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:19:11.593280 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:19:11.626444 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58bdf926-1688-4e18-ac91-2971efb1c695 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:19:11.639413 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e82b785-fb62-4942-93b8-30655d939b1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:19:11.640224 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e82b785-fb62-4942-93b8-30655d939b1f None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:11.640510 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e82b785-fb62-4942-93b8-30655d939b1f None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:11.640790 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e82b785-fb62-4942-93b8-30655d939b1f None admin] PATH_INFO: `/v3/users/nova` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:11.651648 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e82b785-fb62-4942-93b8-30655d939b1f None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:19:11.654550 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e82b785-fb62-4942-93b8-30655d939b1f None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:19:11.676071 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application [None req-7e82b785-fb62-4942-93b8-30655d939b1f None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 15 10:19:11.676071 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:19:11.676071 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:19:11.676071 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:19:11.676071 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:19:11.676071 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:19:11.676071 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:19:11.676071 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:19:11.676071 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:19:11.676071 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:19:11.676071 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:19:11.676071 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:19:11.676071 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 15 10:19:11.676071 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 15 10:19:11.676071 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 15 10:19:11.676071 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 15 10:19:11.677063 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:19:11.677063 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:19:11.677063 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 15 10:19:11.677063 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 10:19:11.677063 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 15 10:19:11.677063 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 10:19:11.677063 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:19:11.677063 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:19:11.677063 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:19:11.677063 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:19:11.677063 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:19:11.677063 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application with Lock( Jun 15 10:19:11.677063 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:19:11.677063 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:19:11.677063 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:19:11.677063 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:19:11.677063 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:19:11.677063 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:19:11.678013 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:19:11.678013 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:19:11.678013 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 15 10:19:11.678013 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 15 10:19:11.678013 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 15 10:19:11.678013 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 15 10:19:11.678013 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 15 10:19:11.678013 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 15 10:19:11.678013 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 15 10:19:11.678013 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application Jun 15 10:19:11.680805 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 395/785] 10.222.0.49 () {56 vars in 1093 bytes} [Wed Jun 15 10:19:11 2022] GET /identity/v3/users/nova => generated 82 bytes in 298 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:19:11.686989 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e82b785-fb62-4942-93b8-30655d939b1f None admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:19:11.702666 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c091bdbc-91ab-4b2f-a096-d5e3e983ce46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:19:11.703744 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c091bdbc-91ab-4b2f-a096-d5e3e983ce46 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:11.704255 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c091bdbc-91ab-4b2f-a096-d5e3e983ce46 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:11.704735 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c091bdbc-91ab-4b2f-a096-d5e3e983ce46 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:11.705735 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c091bdbc-91ab-4b2f-a096-d5e3e983ce46 None admin] RBAC: Adding query filter params (interface=public, service_id=59bf8a0869ab4089a642c29dc13c40a5, region_id=RegionOne) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:19:11.706250 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c091bdbc-91ab-4b2f-a096-d5e3e983ce46 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:19:11.707853 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c091bdbc-91ab-4b2f-a096-d5e3e983ce46 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:19:11.720469 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 391/786] 10.222.0.49 () {56 vars in 1251 bytes} [Wed Jun 15 10:19:11 2022] GET /identity/v3/endpoints?service_id=59bf8a0869ab4089a642c29dc13c40a5&interface=public®ion_id=RegionOne => generated 194 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 10:19:11.725390 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c091bdbc-91ab-4b2f-a096-d5e3e983ce46 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:19:11.787496 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=100 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:19:11.875336 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e82b785-fb62-4942-93b8-30655d939b1f None admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:19:11.910847 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c091bdbc-91ab-4b2f-a096-d5e3e983ce46 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:19:11.936322 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae2f127-5297-400f-94d1-1d7c46b37a94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:19:11.936978 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae2f127-5297-400f-94d1-1d7c46b37a94 None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:11.937267 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae2f127-5297-400f-94d1-1d7c46b37a94 None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:11.937544 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae2f127-5297-400f-94d1-1d7c46b37a94 None admin] PATH_INFO: `/v3/users` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:11.938389 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae2f127-5297-400f-94d1-1d7c46b37a94 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:19:11.938702 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae2f127-5297-400f-94d1-1d7c46b37a94 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:19:11.940197 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae2f127-5297-400f-94d1-1d7c46b37a94 None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:19:11.970424 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 396/787] 10.222.0.49 () {56 vars in 1133 bytes} [Wed Jun 15 10:19:11 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:19:11.980559 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae2f127-5297-400f-94d1-1d7c46b37a94 None admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:19:11.981523 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ef2d49-c1c4-4990-a380-69216f3ef85b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:19:11.982217 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ef2d49-c1c4-4990-a380-69216f3ef85b None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:11.982506 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ef2d49-c1c4-4990-a380-69216f3ef85b None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:11.982786 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ef2d49-c1c4-4990-a380-69216f3ef85b None admin] PATH_INFO: `/v3/services` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:11.983833 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ef2d49-c1c4-4990-a380-69216f3ef85b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:19:11.985247 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ef2d49-c1c4-4990-a380-69216f3ef85b None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:19:11.997128 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 392/788] 10.222.0.49 () {56 vars in 1087 bytes} [Wed Jun 15 10:19:11 2022] GET /identity/v3/services => generated 772 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 10:19:12.112666 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2632e6fc-17f9-4151-907b-9b7873e37cd2 None None] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:12.112985 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2632e6fc-17f9-4151-907b-9b7873e37cd2 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:12.114202 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2632e6fc-17f9-4151-907b-9b7873e37cd2 None None] PATH_INFO: `/` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:12.115608 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 393/789] 10.222.0.49 () {54 vars in 926 bytes} [Wed Jun 15 10:19:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 15 10:19:12.118336 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae2f127-5297-400f-94d1-1d7c46b37a94 None admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:19:12.122663 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=100 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:19:12.126179 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54edc14d-8a45-4ada-bdfe-4d7b63dc6a90 None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:12.126793 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54edc14d-8a45-4ada-bdfe-4d7b63dc6a90 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:12.127320 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54edc14d-8a45-4ada-bdfe-4d7b63dc6a90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:12.167271 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-54edc14d-8a45-4ada-bdfe-4d7b63dc6a90 None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:19:12.168584 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-54edc14d-8a45-4ada-bdfe-4d7b63dc6a90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:19:12.171467 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:19:12.194990 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc07429-d2c2-4908-8b3f-8829507612f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:19:12.198341 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc07429-d2c2-4908-8b3f-8829507612f4 None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:12.198632 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc07429-d2c2-4908-8b3f-8829507612f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:12.198906 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc07429-d2c2-4908-8b3f-8829507612f4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:12.282736 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 394/790] 10.222.0.49 () {58 vars in 1023 bytes} [Wed Jun 15 10:19:12 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 10:19:12.284105 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a90496-acb4-481d-81b4-307a9e2708e1 None None] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:12.284393 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a90496-acb4-481d-81b4-307a9e2708e1 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:12.284668 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a90496-acb4-481d-81b4-307a9e2708e1 None None] PATH_INFO: `/` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:12.285925 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 395/791] 10.222.0.49 () {54 vars in 926 bytes} [Wed Jun 15 10:19:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:19:12.292538 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc226bc2-3500-481c-aa3b-1836556f4c7b None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:12.292992 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc226bc2-3500-481c-aa3b-1836556f4c7b None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:12.293279 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc226bc2-3500-481c-aa3b-1836556f4c7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:12.296222 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc07429-d2c2-4908-8b3f-8829507612f4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:19:12.297748 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc07429-d2c2-4908-8b3f-8829507612f4 None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:19:12.314513 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-fc226bc2-3500-481c-aa3b-1836556f4c7b None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:19:12.315908 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-fc226bc2-3500-481c-aa3b-1836556f4c7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:19:12.402548 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 397/792] 10.222.0.49 () {58 vars in 1282 bytes} [Wed Jun 15 10:19:11 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 422 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 15 10:19:12.403468 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc07429-d2c2-4908-8b3f-8829507612f4 None admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:19:12.407444 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=100 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:19:12.410431 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 396/793] 10.222.0.49 () {58 vars in 1023 bytes} [Wed Jun 15 10:19:12 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 15 10:19:12.411399 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc226bc2-3500-481c-aa3b-1836556f4c7b None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:19:12.512480 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:19:12.528179 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc07429-d2c2-4908-8b3f-8829507612f4 None admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:19:12.561388 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc226bc2-3500-481c-aa3b-1836556f4c7b None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:19:12.566243 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d4e46c-050d-49d0-b0c5-8e435508898f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:19:12.567192 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d4e46c-050d-49d0-b0c5-8e435508898f None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:12.567573 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d4e46c-050d-49d0-b0c5-8e435508898f None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:12.567944 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d4e46c-050d-49d0-b0c5-8e435508898f None admin] PATH_INFO: `/v3/roles/service` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:12.580374 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d4e46c-050d-49d0-b0c5-8e435508898f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 10:19:12.588828 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application [None req-61d4e46c-050d-49d0-b0c5-8e435508898f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 15 10:19:12.588828 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:19:12.588828 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:19:12.588828 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:19:12.588828 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:19:12.588828 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:19:12.588828 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:19:12.588828 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:19:12.588828 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:19:12.588828 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:19:12.588828 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:19:12.588828 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:19:12.588828 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 10:19:12.588828 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 10:19:12.588828 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 10:19:12.588828 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 10:19:12.589749 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 10:19:12.589749 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 10:19:12.589749 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 10:19:12.589749 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 10:19:12.589749 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:19:12.589749 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:19:12.589749 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:19:12.589749 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:19:12.589749 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:19:12.589749 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:19:12.589749 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:19:12.589749 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application with Lock( Jun 15 10:19:12.589749 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:19:12.589749 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:19:12.589749 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:19:12.589749 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:19:12.590594 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:19:12.590594 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:19:12.590594 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:19:12.590594 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:19:12.590594 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 10:19:12.590594 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 10:19:12.590594 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 10:19:12.590594 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 10:19:12.590594 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 10:19:12.590594 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 10:19:12.590594 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 15 10:19:12.590594 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application Jun 15 10:19:12.590594 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 398/794] 10.222.0.49 () {56 vars in 1102 bytes} [Wed Jun 15 10:19:12 2022] GET /identity/v3/roles/service => generated 85 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:19:12.591327 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d4e46c-050d-49d0-b0c5-8e435508898f None admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:19:12.608961 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03c389a-31c6-49b8-83a4-a262a9b34d38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:19:12.611148 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03c389a-31c6-49b8-83a4-a262a9b34d38 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:12.611595 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03c389a-31c6-49b8-83a4-a262a9b34d38 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:12.612020 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03c389a-31c6-49b8-83a4-a262a9b34d38 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:12.626963 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03c389a-31c6-49b8-83a4-a262a9b34d38 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:19:12.629329 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03c389a-31c6-49b8-83a4-a262a9b34d38 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:19:12.654378 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application [None req-e03c389a-31c6-49b8-83a4-a262a9b34d38 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 10:19:12.654378 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:19:12.654378 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:19:12.654378 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:19:12.654378 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:19:12.654378 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:19:12.654378 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:19:12.654378 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:19:12.654378 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:19:12.654378 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:19:12.654378 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:19:12.654378 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:19:12.654378 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 10:19:12.654378 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 10:19:12.654378 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 10:19:12.655529 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 10:19:12.655529 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:19:12.655529 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:19:12.655529 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:19:12.655529 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:19:12.655529 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:19:12.655529 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:19:12.655529 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:19:12.655529 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application with Lock( Jun 15 10:19:12.655529 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:19:12.655529 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:19:12.655529 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:19:12.655529 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:19:12.655529 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:19:12.655529 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:19:12.655529 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:19:12.655529 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:19:12.656677 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 10:19:12.656677 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 10:19:12.656677 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 10:19:12.656677 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Jun 15 10:19:12.659154 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 397/795] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Jun 15 10:19:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:19:12.660086 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03c389a-31c6-49b8-83a4-a262a9b34d38 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:19:12.666407 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=100 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:19:12.722209 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d4e46c-050d-49d0-b0c5-8e435508898f None admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:19:12.765683 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5683b4d1-b444-49aa-b12e-cc2dc259f335 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:19:12.766545 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5683b4d1-b444-49aa-b12e-cc2dc259f335 None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:12.766922 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5683b4d1-b444-49aa-b12e-cc2dc259f335 None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:12.767336 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5683b4d1-b444-49aa-b12e-cc2dc259f335 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:12.775489 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03c389a-31c6-49b8-83a4-a262a9b34d38 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:19:12.779364 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5683b4d1-b444-49aa-b12e-cc2dc259f335 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 10:19:12.787206 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application [None req-5683b4d1-b444-49aa-b12e-cc2dc259f335 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 10:19:12.787206 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:19:12.787206 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:19:12.787206 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:19:12.787206 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:19:12.787206 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:19:12.787206 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:19:12.787206 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:19:12.787206 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:19:12.787206 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:19:12.787206 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:19:12.787206 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:19:12.787206 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 10:19:12.787206 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 10:19:12.787206 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 10:19:12.787206 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 10:19:12.788411 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 10:19:12.788411 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 10:19:12.788411 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 10:19:12.788411 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 10:19:12.788411 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:19:12.788411 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:19:12.788411 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:19:12.788411 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:19:12.788411 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:19:12.788411 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:19:12.788411 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:19:12.788411 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application with Lock( Jun 15 10:19:12.788411 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:19:12.788411 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:19:12.788411 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:19:12.788411 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:19:12.789503 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:19:12.789503 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:19:12.789503 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:19:12.789503 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:19:12.789503 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 10:19:12.789503 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 10:19:12.789503 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 10:19:12.789503 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 10:19:12.789503 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 10:19:12.789503 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 10:19:12.789503 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 10:19:12.789503 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application Jun 15 10:19:12.796392 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 399/796] 10.222.0.49 () {56 vars in 1096 bytes} [Wed Jun 15 10:19:12 2022] GET /identity/v3/roles/admin => generated 83 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:19:12.796392 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5683b4d1-b444-49aa-b12e-cc2dc259f335 None admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:19:12.808722 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:19:12.828050 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8858174-f4dc-420d-9fc2-8c85bb9323cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:19:12.828737 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8858174-f4dc-420d-9fc2-8c85bb9323cf None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:12.829026 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8858174-f4dc-420d-9fc2-8c85bb9323cf None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:12.829340 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8858174-f4dc-420d-9fc2-8c85bb9323cf None admin] PATH_INFO: `/v3/roles` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:12.830699 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8858174-f4dc-420d-9fc2-8c85bb9323cf None admin] RBAC: Adding query filter params (name=service) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:19:12.831173 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8858174-f4dc-420d-9fc2-8c85bb9323cf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:19:12.832808 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8858174-f4dc-420d-9fc2-8c85bb9323cf None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:19:12.844892 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 398/797] 10.222.0.49 () {56 vars in 1103 bytes} [Wed Jun 15 10:19:12 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:19:12.845799 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8858174-f4dc-420d-9fc2-8c85bb9323cf None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:19:12.930664 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5683b4d1-b444-49aa-b12e-cc2dc259f335 None admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:19:12.954853 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=100 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:19:12.966942 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5010c152-3a54-442b-84b4-63054b3553eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:19:12.967638 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5010c152-3a54-442b-84b4-63054b3553eb None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:12.967923 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5010c152-3a54-442b-84b4-63054b3553eb None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:12.968200 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5010c152-3a54-442b-84b4-63054b3553eb None admin] PATH_INFO: `/v3/domains` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:12.968763 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8858174-f4dc-420d-9fc2-8c85bb9323cf None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:19:12.968931 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5010c152-3a54-442b-84b4-63054b3553eb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:19:12.969220 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5010c152-3a54-442b-84b4-63054b3553eb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:19:12.970581 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5010c152-3a54-442b-84b4-63054b3553eb None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:19:12.982784 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 400/798] 10.222.0.49 () {56 vars in 1109 bytes} [Wed Jun 15 10:19:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:19:12.983625 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5010c152-3a54-442b-84b4-63054b3553eb None admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:19:13.008512 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8911afa7-0aec-4b7c-88ce-c55ac91d8d4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:19:13.009196 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8911afa7-0aec-4b7c-88ce-c55ac91d8d4f None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:13.009537 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8911afa7-0aec-4b7c-88ce-c55ac91d8d4f None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:13.009833 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8911afa7-0aec-4b7c-88ce-c55ac91d8d4f None admin] PATH_INFO: `/v3/roles` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:13.010663 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8911afa7-0aec-4b7c-88ce-c55ac91d8d4f None admin] RBAC: Adding query filter params (name=admin) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:19:13.010976 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8911afa7-0aec-4b7c-88ce-c55ac91d8d4f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:19:13.012401 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8911afa7-0aec-4b7c-88ce-c55ac91d8d4f None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:19:13.025609 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 399/799] 10.222.0.49 () {56 vars in 1099 bytes} [Wed Jun 15 10:19:12 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:19:13.025609 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc51aa97-5048-4b32-aa36-353e328ab392 None None] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:13.025609 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc51aa97-5048-4b32-aa36-353e328ab392 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:13.025609 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc51aa97-5048-4b32-aa36-353e328ab392 None None] PATH_INFO: `/` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:13.025609 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 400/800] 10.222.0.49 () {54 vars in 926 bytes} [Wed Jun 15 10:19:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:19:13.025609 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc51aa97-5048-4b32-aa36-353e328ab392 None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:19:13.074024 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:19:13.098304 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5010c152-3a54-442b-84b4-63054b3553eb None admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:19:13.133450 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc51aa97-5048-4b32-aa36-353e328ab392 None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:19:13.134875 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a7677e-1a8e-40c9-8c31-d2202e7ce50e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:19:13.135700 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a7677e-1a8e-40c9-8c31-d2202e7ce50e None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:13.136592 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a7677e-1a8e-40c9-8c31-d2202e7ce50e None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:13.136592 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a7677e-1a8e-40c9-8c31-d2202e7ce50e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:13.146622 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a7677e-1a8e-40c9-8c31-d2202e7ce50e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:19:13.148328 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a7677e-1a8e-40c9-8c31-d2202e7ce50e None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:19:13.168662 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edbf0636-acaf-4555-bacd-7af39409c344 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:19:13.169078 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application [None req-06a7677e-1a8e-40c9-8c31-d2202e7ce50e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 10:19:13.169078 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:19:13.169078 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:19:13.169078 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:19:13.169078 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:19:13.169078 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:19:13.169078 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:19:13.169078 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:19:13.169078 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:19:13.169078 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:19:13.169078 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:19:13.169078 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:19:13.169078 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 10:19:13.169078 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 10:19:13.169078 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 10:19:13.169929 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 10:19:13.169929 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:19:13.169929 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:19:13.169929 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:19:13.169929 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:19:13.169929 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:19:13.169929 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:19:13.169929 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:19:13.169929 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application with Lock( Jun 15 10:19:13.169929 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:19:13.169929 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:19:13.169929 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:19:13.169929 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:19:13.169929 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:19:13.169929 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:19:13.169929 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:19:13.169929 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:19:13.170929 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 10:19:13.170929 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 10:19:13.170929 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 10:19:13.170929 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application Jun 15 10:19:13.171183 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbf0636-acaf-4555-bacd-7af39409c344 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:13.171183 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbf0636-acaf-4555-bacd-7af39409c344 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:13.171183 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbf0636-acaf-4555-bacd-7af39409c344 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:13.175290 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 401/801] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Jun 15 10:19:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:19:13.176309 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a7677e-1a8e-40c9-8c31-d2202e7ce50e None admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:19:13.180023 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edbf0636-acaf-4555-bacd-7af39409c344 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:19:13.181466 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edbf0636-acaf-4555-bacd-7af39409c344 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:19:13.196896 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application [None req-edbf0636-acaf-4555-bacd-7af39409c344 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 10:19:13.196896 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:19:13.196896 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:19:13.196896 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:19:13.196896 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:19:13.196896 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:19:13.196896 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:19:13.196896 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:19:13.196896 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:19:13.196896 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:19:13.196896 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:19:13.196896 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:19:13.196896 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 15 10:19:13.196896 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 15 10:19:13.196896 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 15 10:19:13.197846 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 15 10:19:13.197846 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:19:13.197846 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:19:13.197846 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:19:13.197846 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:19:13.197846 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:19:13.197846 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:19:13.197846 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:19:13.197846 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application with Lock( Jun 15 10:19:13.197846 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:19:13.197846 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:19:13.197846 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:19:13.197846 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:19:13.197846 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:19:13.197846 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:19:13.197846 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:19:13.197846 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:19:13.198779 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 15 10:19:13.198779 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 15 10:19:13.198779 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 15 10:19:13.198779 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 15 10:19:13.198779 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 15 10:19:13.198779 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 15 10:19:13.198779 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 10:19:13.198779 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Jun 15 10:19:13.198779 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 401/802] 10.222.0.49 () {56 vars in 1111 bytes} [Wed Jun 15 10:19:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:19:13.199294 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f7ee42-aa95-4b7a-a3e4-34586703adf1 None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:13.199858 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f7ee42-aa95-4b7a-a3e4-34586703adf1 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:13.200138 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f7ee42-aa95-4b7a-a3e4-34586703adf1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:13.216999 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=100 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:19:13.217797 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-06f7ee42-aa95-4b7a-a3e4-34586703adf1 None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:19:13.219134 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-06f7ee42-aa95-4b7a-a3e4-34586703adf1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:19:13.291853 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 402/803] 10.222.0.49 () {58 vars in 1023 bytes} [Wed Jun 15 10:19:13 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 10:19:13.291853 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f7ee42-aa95-4b7a-a3e4-34586703adf1 None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:19:13.303256 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a7677e-1a8e-40c9-8c31-d2202e7ce50e None admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:19:13.336397 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:19:13.339902 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f2af3c-16bb-4ff2-8b32-de7471a95a39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:19:13.340711 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f2af3c-16bb-4ff2-8b32-de7471a95a39 None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:13.340999 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f2af3c-16bb-4ff2-8b32-de7471a95a39 None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:13.341278 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f2af3c-16bb-4ff2-8b32-de7471a95a39 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:13.351572 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f2af3c-16bb-4ff2-8b32-de7471a95a39 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:19:13.353042 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f2af3c-16bb-4ff2-8b32-de7471a95a39 None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:19:13.374941 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application [None req-06f2af3c-16bb-4ff2-8b32-de7471a95a39 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 10:19:13.374941 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:19:13.374941 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:19:13.374941 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:19:13.374941 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:19:13.374941 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:19:13.374941 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:19:13.374941 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:19:13.374941 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:19:13.374941 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:19:13.374941 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:19:13.374941 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:19:13.374941 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 10:19:13.374941 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 10:19:13.374941 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 10:19:13.376069 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 10:19:13.376069 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:19:13.376069 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:19:13.376069 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:19:13.376069 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:19:13.376069 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:19:13.376069 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:19:13.376069 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:19:13.376069 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application with Lock( Jun 15 10:19:13.376069 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:19:13.376069 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:19:13.376069 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:19:13.376069 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:19:13.376069 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:19:13.376069 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:19:13.376069 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:19:13.376069 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:19:13.376968 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 10:19:13.376968 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 10:19:13.376968 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 10:19:13.376968 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application Jun 15 10:19:13.377852 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 402/804] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Jun 15 10:19:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:19:13.378681 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f2af3c-16bb-4ff2-8b32-de7471a95a39 None admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:19:13.412153 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f7ee42-aa95-4b7a-a3e4-34586703adf1 None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:19:13.446581 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd35882b-ebb2-4f2b-bdfc-9a2bd969274d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:19:13.447285 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd35882b-ebb2-4f2b-bdfc-9a2bd969274d None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:13.447746 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd35882b-ebb2-4f2b-bdfc-9a2bd969274d None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:13.448031 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd35882b-ebb2-4f2b-bdfc-9a2bd969274d None admin] PATH_INFO: `/v3/domains` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:13.448748 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd35882b-ebb2-4f2b-bdfc-9a2bd969274d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:19:13.449134 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd35882b-ebb2-4f2b-bdfc-9a2bd969274d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:19:13.450492 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd35882b-ebb2-4f2b-bdfc-9a2bd969274d None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:19:13.461767 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 403/805] 10.222.0.49 () {56 vars in 1109 bytes} [Wed Jun 15 10:19:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:19:13.462549 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd35882b-ebb2-4f2b-bdfc-9a2bd969274d None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:19:13.476869 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=100 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:19:13.494229 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f2af3c-16bb-4ff2-8b32-de7471a95a39 None admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:19:13.531137 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9f37b2-588b-4c7d-af83-2f9460956d91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:19:13.531782 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9f37b2-588b-4c7d-af83-2f9460956d91 None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:13.532063 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9f37b2-588b-4c7d-af83-2f9460956d91 None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:13.532340 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9f37b2-588b-4c7d-af83-2f9460956d91 None admin] PATH_INFO: `/v3/projects` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:13.533119 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd9f37b2-588b-4c7d-af83-2f9460956d91 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:19:13.533457 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd9f37b2-588b-4c7d-af83-2f9460956d91 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:19:13.534884 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd9f37b2-588b-4c7d-af83-2f9460956d91 None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:19:13.546917 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 403/806] 10.222.0.49 () {56 vars in 1148 bytes} [Wed Jun 15 10:19:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:19:13.547681 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9f37b2-588b-4c7d-af83-2f9460956d91 None admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:19:13.567372 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:19:13.575219 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd35882b-ebb2-4f2b-bdfc-9a2bd969274d None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:19:13.611014 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c89b37d-a040-4028-8917-3223a253489f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:19:13.611890 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c89b37d-a040-4028-8917-3223a253489f None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:13.612183 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c89b37d-a040-4028-8917-3223a253489f None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:13.612563 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c89b37d-a040-4028-8917-3223a253489f None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:13.613290 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c89b37d-a040-4028-8917-3223a253489f None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:19:13.614699 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c89b37d-a040-4028-8917-3223a253489f None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:19:13.628045 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application [None req-3c89b37d-a040-4028-8917-3223a253489f None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 15 10:19:13.628045 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:19:13.628045 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 15 10:19:13.628045 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 15 10:19:13.628045 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 15 10:19:13.628045 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 15 10:19:13.628045 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 15 10:19:13.628045 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 10:19:13.628045 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 15 10:19:13.628045 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Jun 15 10:19:13.628045 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 15 10:19:13.628045 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Jun 15 10:19:13.628045 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:19:13.628045 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:19:13.628045 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:19:13.628045 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:19:13.628971 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:19:13.628971 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:19:13.628971 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:19:13.628971 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:19:13.628971 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:19:13.628971 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:19:13.628971 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:19:13.628971 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 15 10:19:13.628971 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 15 10:19:13.628971 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 15 10:19:13.628971 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 15 10:19:13.628971 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:19:13.628971 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:19:13.628971 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:19:13.628971 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:19:13.628971 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:19:13.629829 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:19:13.629829 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:19:13.629829 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application with Lock( Jun 15 10:19:13.629829 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:19:13.629829 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:19:13.629829 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:19:13.629829 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:19:13.629829 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:19:13.629829 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:19:13.629829 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:19:13.629829 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:19:13.629829 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 15 10:19:13.629829 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 10:19:13.629829 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 15 10:19:13.629829 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Jun 15 10:19:13.629829 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 404/807] 10.222.0.49 () {56 vars in 1129 bytes} [Wed Jun 15 10:19:13 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:19:13.630670 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c89b37d-a040-4028-8917-3223a253489f None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:19:13.653005 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9f37b2-588b-4c7d-af83-2f9460956d91 None admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:19:13.688270 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b52caa-b676-4850-a81e-d0adeb1b512b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:19:13.688932 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b52caa-b676-4850-a81e-d0adeb1b512b None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:13.689213 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b52caa-b676-4850-a81e-d0adeb1b512b None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:13.689486 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b52caa-b676-4850-a81e-d0adeb1b512b None admin] PATH_INFO: `/v3/domains` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:13.690197 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b52caa-b676-4850-a81e-d0adeb1b512b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:19:13.690497 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b52caa-b676-4850-a81e-d0adeb1b512b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:19:13.691896 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b52caa-b676-4850-a81e-d0adeb1b512b None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:19:13.703837 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 404/808] 10.222.0.49 () {56 vars in 1109 bytes} [Wed Jun 15 10:19:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:19:13.704987 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b52caa-b676-4850-a81e-d0adeb1b512b None admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:19:13.714663 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=100 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:19:13.740701 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c89b37d-a040-4028-8917-3223a253489f None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:19:13.778439 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b612c0-a71e-47c7-b5f0-d12acba8e550 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:19:13.779293 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b612c0-a71e-47c7-b5f0-d12acba8e550 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:13.779583 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b612c0-a71e-47c7-b5f0-d12acba8e550 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:13.779869 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b612c0-a71e-47c7-b5f0-d12acba8e550 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:13.790796 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b612c0-a71e-47c7-b5f0-d12acba8e550 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:19:13.790796 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b612c0-a71e-47c7-b5f0-d12acba8e550 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:19:13.804605 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application [None req-b9b612c0-a71e-47c7-b5f0-d12acba8e550 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 15 10:19:13.804605 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:19:13.804605 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:19:13.804605 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:19:13.804605 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:19:13.804605 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:19:13.804605 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:19:13.804605 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:19:13.804605 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:19:13.804605 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:19:13.804605 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:19:13.804605 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:19:13.804605 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 15 10:19:13.804605 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 15 10:19:13.804605 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 15 10:19:13.804605 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 15 10:19:13.805541 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:19:13.805541 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:19:13.805541 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 15 10:19:13.805541 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 10:19:13.805541 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 15 10:19:13.805541 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 10:19:13.805541 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:19:13.805541 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:19:13.805541 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:19:13.805541 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:19:13.805541 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:19:13.805541 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application with Lock( Jun 15 10:19:13.805541 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:19:13.805541 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:19:13.805541 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:19:13.805541 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:19:13.805541 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:19:13.805541 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:19:13.807316 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:19:13.807316 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:19:13.807316 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 15 10:19:13.807316 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 15 10:19:13.807316 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 15 10:19:13.807316 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 15 10:19:13.807316 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 15 10:19:13.807316 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 15 10:19:13.807316 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 15 10:19:13.807316 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Jun 15 10:19:13.807316 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 405/809] 10.222.0.49 () {56 vars in 1117 bytes} [Wed Jun 15 10:19:13 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:19:13.807316 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b612c0-a71e-47c7-b5f0-d12acba8e550 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:19:13.821939 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b52caa-b676-4850-a81e-d0adeb1b512b None admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:19:13.828885 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:19:13.859204 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f8df62-551f-4eb9-b2ce-16bf5fc30bab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:19:13.860088 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f8df62-551f-4eb9-b2ce-16bf5fc30bab None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:13.860523 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f8df62-551f-4eb9-b2ce-16bf5fc30bab None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:13.860804 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f8df62-551f-4eb9-b2ce-16bf5fc30bab None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:13.861628 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f8df62-551f-4eb9-b2ce-16bf5fc30bab None admin] RBAC: Adding query filter params (role.id=8051174e7a7947d5b991ebd793c2e543, scope.project.id=559b3e713f5844b6ae37151d56dd419f, user.id=5ac54528370d4747b6e8777fba9d0c82) {{(pid=80119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:19:13.861939 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f8df62-551f-4eb9-b2ce-16bf5fc30bab None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:19:13.863411 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f8df62-551f-4eb9-b2ce-16bf5fc30bab None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:19:13.869879 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 405/810] 10.222.0.49 () {56 vars in 1374 bytes} [Wed Jun 15 10:19:13 2022] GET /identity/v3/role_assignments?user.id=5ac54528370d4747b6e8777fba9d0c82&scope.project.id=559b3e713f5844b6ae37151d56dd419f&role.id=8051174e7a7947d5b991ebd793c2e543 => generated 608 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:19:13.870746 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f8df62-551f-4eb9-b2ce-16bf5fc30bab None admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:19:13.931117 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b612c0-a71e-47c7-b5f0-d12acba8e550 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:19:13.971070 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198a6a24-e270-44ec-89c1-fdf0d89d888e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:19:13.971976 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=100 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:19:13.973115 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198a6a24-e270-44ec-89c1-fdf0d89d888e None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:13.973668 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198a6a24-e270-44ec-89c1-fdf0d89d888e None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:13.973980 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198a6a24-e270-44ec-89c1-fdf0d89d888e None admin] PATH_INFO: `/v3/services` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:13.974802 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-198a6a24-e270-44ec-89c1-fdf0d89d888e None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:19:13.975139 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-198a6a24-e270-44ec-89c1-fdf0d89d888e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:19:13.976745 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-198a6a24-e270-44ec-89c1-fdf0d89d888e None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:19:13.987477 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 406/811] 10.222.0.49 () {56 vars in 1124 bytes} [Wed Jun 15 10:19:13 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:19:13.987477 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198a6a24-e270-44ec-89c1-fdf0d89d888e None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:19:14.022684 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f8df62-551f-4eb9-b2ce-16bf5fc30bab None admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:19:14.076723 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b15337-6292-48ef-bccd-abb88f5bd400 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:19:14.077485 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b15337-6292-48ef-bccd-abb88f5bd400 None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:14.077744 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b15337-6292-48ef-bccd-abb88f5bd400 None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:14.078026 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b15337-6292-48ef-bccd-abb88f5bd400 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:14.086369 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b15337-6292-48ef-bccd-abb88f5bd400 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:19:14.087866 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b15337-6292-48ef-bccd-abb88f5bd400 None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:19:14.093036 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:19:14.105780 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application [None req-25b15337-6292-48ef-bccd-abb88f5bd400 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 15 10:19:14.105780 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:19:14.105780 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:19:14.105780 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:19:14.105780 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:19:14.105780 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:19:14.105780 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:19:14.105780 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:19:14.105780 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:19:14.105780 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:19:14.105780 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:19:14.105780 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:19:14.105780 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 15 10:19:14.105780 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 15 10:19:14.105780 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 15 10:19:14.105780 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 15 10:19:14.107042 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:19:14.107042 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:19:14.107042 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 15 10:19:14.107042 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 10:19:14.107042 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 15 10:19:14.107042 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 10:19:14.107042 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:19:14.107042 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:19:14.107042 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:19:14.107042 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:19:14.107042 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:19:14.107042 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application with Lock( Jun 15 10:19:14.107042 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:19:14.107042 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:19:14.107042 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:19:14.107042 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:19:14.107042 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:19:14.107042 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:19:14.107997 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:19:14.107997 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:19:14.107997 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 15 10:19:14.107997 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 15 10:19:14.107997 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 15 10:19:14.107997 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 15 10:19:14.107997 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 15 10:19:14.107997 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 15 10:19:14.107997 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 15 10:19:14.107997 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application Jun 15 10:19:14.107997 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 406/812] 10.222.0.49 () {56 vars in 1096 bytes} [Wed Jun 15 10:19:13 2022] GET /identity/v3/users/swift => generated 83 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:19:14.107997 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b15337-6292-48ef-bccd-abb88f5bd400 None admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:19:14.123845 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198a6a24-e270-44ec-89c1-fdf0d89d888e None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:19:14.166334 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25e760c-26ab-4449-aef5-777262beef9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:19:14.167691 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25e760c-26ab-4449-aef5-777262beef9a None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:14.168146 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25e760c-26ab-4449-aef5-777262beef9a None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:14.168804 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25e760c-26ab-4449-aef5-777262beef9a None admin] PATH_INFO: `/v3/users` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:14.170270 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f25e760c-26ab-4449-aef5-777262beef9a None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:19:14.170993 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f25e760c-26ab-4449-aef5-777262beef9a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:19:14.173253 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f25e760c-26ab-4449-aef5-777262beef9a None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:19:14.198229 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 407/813] 10.222.0.49 () {56 vars in 1149 bytes} [Wed Jun 15 10:19:13 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:19:14.199507 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25e760c-26ab-4449-aef5-777262beef9a None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:19:14.234927 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b15337-6292-48ef-bccd-abb88f5bd400 None admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:19:14.265878 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=100 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:19:14.272209 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6968b11-cfc3-47c6-911a-1884711ec0e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:19:14.272890 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6968b11-cfc3-47c6-911a-1884711ec0e2 None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:14.273174 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6968b11-cfc3-47c6-911a-1884711ec0e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:14.273448 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6968b11-cfc3-47c6-911a-1884711ec0e2 None admin] PATH_INFO: `/v3/services` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:14.274149 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6968b11-cfc3-47c6-911a-1884711ec0e2 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:19:14.274454 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6968b11-cfc3-47c6-911a-1884711ec0e2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:19:14.276177 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6968b11-cfc3-47c6-911a-1884711ec0e2 None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:19:14.281761 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 407/814] 10.222.0.49 () {56 vars in 1124 bytes} [Wed Jun 15 10:19:14 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:19:14.282548 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6968b11-cfc3-47c6-911a-1884711ec0e2 None admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:19:14.338475 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25e760c-26ab-4449-aef5-777262beef9a None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:19:14.376564 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd57bfc-e806-4678-aa25-dafd5d912e98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:19:14.377234 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd57bfc-e806-4678-aa25-dafd5d912e98 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:14.377514 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd57bfc-e806-4678-aa25-dafd5d912e98 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:14.377787 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd57bfc-e806-4678-aa25-dafd5d912e98 None admin] PATH_INFO: `/v3/users` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:14.378612 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd57bfc-e806-4678-aa25-dafd5d912e98 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:19:14.378919 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd57bfc-e806-4678-aa25-dafd5d912e98 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:19:14.380368 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd57bfc-e806-4678-aa25-dafd5d912e98 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:19:14.384920 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6968b11-cfc3-47c6-911a-1884711ec0e2 None admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:19:14.391461 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:19:14.402288 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 408/815] 10.222.0.49 () {56 vars in 1135 bytes} [Wed Jun 15 10:19:14 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:19:14.403620 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c71d0ab-821d-4a05-9934-347a4f1e7ddd None None] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:14.403907 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c71d0ab-821d-4a05-9934-347a4f1e7ddd None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:14.404181 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c71d0ab-821d-4a05-9934-347a4f1e7ddd None None] PATH_INFO: `/` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:14.405582 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 409/816] 10.222.0.49 () {54 vars in 926 bytes} [Wed Jun 15 10:19:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:19:14.409987 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c71d0ab-821d-4a05-9934-347a4f1e7ddd None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:19:14.425236 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7539fc98-5e87-4813-821e-4d2063b0005e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:19:14.426037 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7539fc98-5e87-4813-821e-4d2063b0005e None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:14.426319 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7539fc98-5e87-4813-821e-4d2063b0005e None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:14.426590 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7539fc98-5e87-4813-821e-4d2063b0005e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:14.437815 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7539fc98-5e87-4813-821e-4d2063b0005e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:19:14.439305 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7539fc98-5e87-4813-821e-4d2063b0005e None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:19:14.456824 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application [None req-7539fc98-5e87-4813-821e-4d2063b0005e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 10:19:14.456824 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:19:14.456824 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:19:14.456824 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:19:14.456824 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:19:14.456824 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:19:14.456824 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:19:14.456824 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:19:14.456824 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:19:14.456824 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:19:14.456824 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:19:14.456824 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:19:14.456824 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 10:19:14.456824 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 10:19:14.456824 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 10:19:14.457757 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 10:19:14.457757 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:19:14.457757 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:19:14.457757 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:19:14.457757 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:19:14.457757 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:19:14.457757 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:19:14.457757 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:19:14.457757 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application with Lock( Jun 15 10:19:14.457757 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:19:14.457757 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:19:14.457757 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:19:14.457757 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:19:14.457757 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:19:14.457757 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:19:14.457757 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:19:14.457757 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:19:14.458661 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 10:19:14.458661 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 10:19:14.458661 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 10:19:14.458661 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application Jun 15 10:19:14.462613 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 408/817] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Jun 15 10:19:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:19:14.463852 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7539fc98-5e87-4813-821e-4d2063b0005e None admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:19:14.512311 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=100 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:19:14.525702 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c71d0ab-821d-4a05-9934-347a4f1e7ddd None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:19:14.561546 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c502fdb-3fec-4252-9fc0-dfdfdc5300d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:19:14.562201 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c502fdb-3fec-4252-9fc0-dfdfdc5300d1 None admin] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:14.562484 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c502fdb-3fec-4252-9fc0-dfdfdc5300d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:14.562759 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c502fdb-3fec-4252-9fc0-dfdfdc5300d1 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:14.563627 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c502fdb-3fec-4252-9fc0-dfdfdc5300d1 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:19:14.565099 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c502fdb-3fec-4252-9fc0-dfdfdc5300d1 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:19:14.566607 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7539fc98-5e87-4813-821e-4d2063b0005e None admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:19:14.591262 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 410/818] 10.222.0.49 () {60 vars in 1144 bytes} [Wed Jun 15 10:19:14 2022] POST /identity/v3/endpoints => generated 346 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 10:19:14.591262 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2789d6c-d38a-4361-8470-edb26122f3f8 None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:14.591262 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2789d6c-d38a-4361-8470-edb26122f3f8 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:14.591262 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2789d6c-d38a-4361-8470-edb26122f3f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:14.613835 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6085cdf0-4528-4f7f-bb91-6d0dcbd556d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:19:14.615102 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6085cdf0-4528-4f7f-bb91-6d0dcbd556d6 None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:14.615616 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6085cdf0-4528-4f7f-bb91-6d0dcbd556d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:14.615982 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6085cdf0-4528-4f7f-bb91-6d0dcbd556d6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:14.618249 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-f2789d6c-d38a-4361-8470-edb26122f3f8 None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:19:14.620636 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-f2789d6c-d38a-4361-8470-edb26122f3f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:19:14.629718 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6085cdf0-4528-4f7f-bb91-6d0dcbd556d6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:19:14.633005 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6085cdf0-4528-4f7f-bb91-6d0dcbd556d6 None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:19:14.651839 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application [None req-6085cdf0-4528-4f7f-bb91-6d0dcbd556d6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 10:19:14.651839 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:19:14.651839 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:19:14.651839 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:19:14.651839 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:19:14.651839 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:19:14.651839 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:19:14.651839 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:19:14.651839 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:19:14.651839 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:19:14.651839 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:19:14.651839 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:19:14.651839 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 10:19:14.651839 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 10:19:14.651839 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 10:19:14.652812 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 10:19:14.652812 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:19:14.652812 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:19:14.652812 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:19:14.652812 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:19:14.652812 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:19:14.652812 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:19:14.652812 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:19:14.652812 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application with Lock( Jun 15 10:19:14.652812 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:19:14.652812 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:19:14.652812 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:19:14.652812 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:19:14.652812 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:19:14.652812 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:19:14.652812 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:19:14.652812 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:19:14.654233 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 10:19:14.654233 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 10:19:14.654233 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 10:19:14.654233 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application Jun 15 10:19:14.655022 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 409/819] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Jun 15 10:19:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:19:14.657895 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6085cdf0-4528-4f7f-bb91-6d0dcbd556d6 None admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:19:14.732457 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:19:14.751976 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 411/820] 10.222.0.49 () {58 vars in 1023 bytes} [Wed Jun 15 10:19:14 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 10:19:14.753096 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2789d6c-d38a-4361-8470-edb26122f3f8 None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:19:14.779920 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=100 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:19:14.791553 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6085cdf0-4528-4f7f-bb91-6d0dcbd556d6 None admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:19:14.840478 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3bfdeb4-4999-4536-963b-872bea7d9d52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:19:14.841152 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3bfdeb4-4999-4536-963b-872bea7d9d52 None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:14.841483 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3bfdeb4-4999-4536-963b-872bea7d9d52 None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:14.841708 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3bfdeb4-4999-4536-963b-872bea7d9d52 None admin] PATH_INFO: `/v3/domains` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:14.842502 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3bfdeb4-4999-4536-963b-872bea7d9d52 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:19:14.842818 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3bfdeb4-4999-4536-963b-872bea7d9d52 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:19:14.844292 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3bfdeb4-4999-4536-963b-872bea7d9d52 None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:19:14.867343 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 410/821] 10.222.0.49 () {56 vars in 1109 bytes} [Wed Jun 15 10:19:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:19:14.868588 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3bfdeb4-4999-4536-963b-872bea7d9d52 None admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:19:14.888909 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2789d6c-d38a-4361-8470-edb26122f3f8 None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:19:14.924059 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878ebfc3-791d-4628-9646-817668070dab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:19:14.925425 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878ebfc3-791d-4628-9646-817668070dab None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:14.925887 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878ebfc3-791d-4628-9646-817668070dab None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:14.926359 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878ebfc3-791d-4628-9646-817668070dab None admin] PATH_INFO: `/v3/domains` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:14.927705 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-878ebfc3-791d-4628-9646-817668070dab None admin] RBAC: Adding query filter params (name=Default) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:19:14.928220 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-878ebfc3-791d-4628-9646-817668070dab None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:19:14.930658 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-878ebfc3-791d-4628-9646-817668070dab None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:19:14.945092 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 412/822] 10.222.0.49 () {56 vars in 1109 bytes} [Wed Jun 15 10:19:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:19:14.945979 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878ebfc3-791d-4628-9646-817668070dab None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:19:14.993253 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3bfdeb4-4999-4536-963b-872bea7d9d52 None admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:19:14.993667 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:19:15.030575 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a242c1f-76ab-496e-bfdd-fa43cfa2e71c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:19:15.031793 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a242c1f-76ab-496e-bfdd-fa43cfa2e71c None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:15.032306 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a242c1f-76ab-496e-bfdd-fa43cfa2e71c None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:15.032800 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a242c1f-76ab-496e-bfdd-fa43cfa2e71c None admin] PATH_INFO: `/v3/services/network` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:15.033647 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a242c1f-76ab-496e-bfdd-fa43cfa2e71c None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:19:15.035248 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a242c1f-76ab-496e-bfdd-fa43cfa2e71c None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:19:15.047484 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application [None req-4a242c1f-76ab-496e-bfdd-fa43cfa2e71c None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jun 15 10:19:15.047484 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:19:15.047484 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 15 10:19:15.047484 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 15 10:19:15.047484 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 15 10:19:15.047484 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 15 10:19:15.047484 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 15 10:19:15.047484 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 10:19:15.047484 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 15 10:19:15.047484 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application Jun 15 10:19:15.047484 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 15 10:19:15.047484 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application Jun 15 10:19:15.047484 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:19:15.047484 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:19:15.047484 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:19:15.047484 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:19:15.047484 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:19:15.048474 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:19:15.048474 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:19:15.048474 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:19:15.048474 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:19:15.048474 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:19:15.048474 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:19:15.048474 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 15 10:19:15.048474 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 15 10:19:15.048474 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 15 10:19:15.048474 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 15 10:19:15.048474 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:19:15.048474 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:19:15.048474 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:19:15.048474 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:19:15.048474 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:19:15.048474 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:19:15.049331 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:19:15.049331 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application with Lock( Jun 15 10:19:15.049331 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:19:15.049331 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:19:15.049331 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:19:15.049331 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:19:15.049331 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:19:15.049331 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:19:15.049331 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:19:15.049331 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:19:15.049331 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 15 10:19:15.049331 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 10:19:15.049331 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 15 10:19:15.049331 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application Jun 15 10:19:15.051825 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878ebfc3-791d-4628-9646-817668070dab None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:19:15.052637 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 411/823] 10.222.0.49 () {56 vars in 1111 bytes} [Wed Jun 15 10:19:14 2022] GET /identity/v3/services/network => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:19:15.053673 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a242c1f-76ab-496e-bfdd-fa43cfa2e71c None admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:19:15.076302 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=100 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:19:15.095727 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4febebed-2182-46a1-a343-7438f0952894 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:19:15.096529 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4febebed-2182-46a1-a343-7438f0952894 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:15.096808 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4febebed-2182-46a1-a343-7438f0952894 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:15.097136 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4febebed-2182-46a1-a343-7438f0952894 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:15.104430 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4febebed-2182-46a1-a343-7438f0952894 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:19:15.105935 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4febebed-2182-46a1-a343-7438f0952894 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:19:15.121201 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application [None req-4febebed-2182-46a1-a343-7438f0952894 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 10:19:15.121201 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:19:15.121201 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:19:15.121201 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:19:15.121201 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:19:15.121201 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:19:15.121201 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:19:15.121201 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:19:15.121201 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:19:15.121201 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:19:15.121201 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:19:15.121201 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:19:15.121201 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 15 10:19:15.121201 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 15 10:19:15.121201 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 15 10:19:15.122131 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 15 10:19:15.122131 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:19:15.122131 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:19:15.122131 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:19:15.122131 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:19:15.122131 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:19:15.122131 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:19:15.122131 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:19:15.122131 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application with Lock( Jun 15 10:19:15.122131 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:19:15.122131 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:19:15.122131 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:19:15.122131 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:19:15.122131 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:19:15.122131 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:19:15.122131 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:19:15.122131 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:19:15.123024 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 15 10:19:15.123024 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 15 10:19:15.123024 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 15 10:19:15.123024 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 15 10:19:15.123024 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 15 10:19:15.123024 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 15 10:19:15.123024 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 10:19:15.123024 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Jun 15 10:19:15.127169 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 413/824] 10.222.0.49 () {56 vars in 1111 bytes} [Wed Jun 15 10:19:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:19:15.128030 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4febebed-2182-46a1-a343-7438f0952894 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:19:15.170198 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a242c1f-76ab-496e-bfdd-fa43cfa2e71c None admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:19:15.206092 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baebfdbf-67c3-4e33-a2ed-f5350ec864c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:19:15.207301 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baebfdbf-67c3-4e33-a2ed-f5350ec864c3 None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:15.207820 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baebfdbf-67c3-4e33-a2ed-f5350ec864c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:15.208297 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baebfdbf-67c3-4e33-a2ed-f5350ec864c3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:15.215750 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baebfdbf-67c3-4e33-a2ed-f5350ec864c3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:19:15.217447 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baebfdbf-67c3-4e33-a2ed-f5350ec864c3 None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:19:15.232662 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application [None req-baebfdbf-67c3-4e33-a2ed-f5350ec864c3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 10:19:15.232662 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:19:15.232662 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:19:15.232662 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:19:15.232662 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:19:15.232662 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:19:15.232662 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:19:15.232662 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:19:15.232662 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:19:15.232662 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:19:15.232662 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:19:15.232662 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:19:15.232662 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 15 10:19:15.232662 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 15 10:19:15.232662 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 15 10:19:15.233578 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 15 10:19:15.233578 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:19:15.233578 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:19:15.233578 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:19:15.233578 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:19:15.233578 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:19:15.233578 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:19:15.233578 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:19:15.233578 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application with Lock( Jun 15 10:19:15.233578 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:19:15.233578 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:19:15.233578 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:19:15.233578 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:19:15.233578 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:19:15.233578 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:19:15.233578 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:19:15.233578 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:19:15.234490 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 15 10:19:15.234490 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 15 10:19:15.234490 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 15 10:19:15.234490 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 15 10:19:15.234490 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 15 10:19:15.234490 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 15 10:19:15.234490 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 10:19:15.234490 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application Jun 15 10:19:15.235387 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4febebed-2182-46a1-a343-7438f0952894 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:19:15.237132 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 412/825] 10.222.0.49 () {56 vars in 1111 bytes} [Wed Jun 15 10:19:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:19:15.237377 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:19:15.238304 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baebfdbf-67c3-4e33-a2ed-f5350ec864c3 None admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:19:15.271468 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c47c7b-1786-44f4-80d5-a6a4c39c33b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:19:15.272130 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c47c7b-1786-44f4-80d5-a6a4c39c33b8 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:15.272412 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c47c7b-1786-44f4-80d5-a6a4c39c33b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:15.272689 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c47c7b-1786-44f4-80d5-a6a4c39c33b8 None admin] PATH_INFO: `/v3/services` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:15.273385 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c47c7b-1786-44f4-80d5-a6a4c39c33b8 None admin] RBAC: Adding query filter params (name=network) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:19:15.273688 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c47c7b-1786-44f4-80d5-a6a4c39c33b8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:19:15.275038 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c47c7b-1786-44f4-80d5-a6a4c39c33b8 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:19:15.284302 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 414/826] 10.222.0.49 () {56 vars in 1112 bytes} [Wed Jun 15 10:19:15 2022] GET /identity/v3/services?name=network => generated 124 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:19:15.285298 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c47c7b-1786-44f4-80d5-a6a4c39c33b8 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:19:15.322270 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=100 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:19:15.344753 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baebfdbf-67c3-4e33-a2ed-f5350ec864c3 None admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:19:15.380301 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e47900d-b681-4a68-84f6-0560d2d35368 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:19:15.381366 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e47900d-b681-4a68-84f6-0560d2d35368 None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:15.381873 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e47900d-b681-4a68-84f6-0560d2d35368 None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:15.382363 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e47900d-b681-4a68-84f6-0560d2d35368 None admin] PATH_INFO: `/v3/projects` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:15.383380 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e47900d-b681-4a68-84f6-0560d2d35368 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:19:15.383891 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e47900d-b681-4a68-84f6-0560d2d35368 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:19:15.385560 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e47900d-b681-4a68-84f6-0560d2d35368 None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:19:15.390393 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c47c7b-1786-44f4-80d5-a6a4c39c33b8 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:19:15.403154 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 413/827] 10.222.0.49 () {56 vars in 1148 bytes} [Wed Jun 15 10:19:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:19:15.404278 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e47900d-b681-4a68-84f6-0560d2d35368 None admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:19:15.427493 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71905462-dd94-47a5-9dfe-ac413130faf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:19:15.428044 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71905462-dd94-47a5-9dfe-ac413130faf0 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:15.428323 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71905462-dd94-47a5-9dfe-ac413130faf0 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:15.428596 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71905462-dd94-47a5-9dfe-ac413130faf0 None admin] PATH_INFO: `/v3/projects` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:15.429447 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71905462-dd94-47a5-9dfe-ac413130faf0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:19:15.429686 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71905462-dd94-47a5-9dfe-ac413130faf0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:19:15.431168 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71905462-dd94-47a5-9dfe-ac413130faf0 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:19:15.444496 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 415/828] 10.222.0.49 () {56 vars in 1148 bytes} [Wed Jun 15 10:19:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:19:15.445289 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71905462-dd94-47a5-9dfe-ac413130faf0 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:19:15.475125 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:19:15.510116 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e47900d-b681-4a68-84f6-0560d2d35368 None admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:19:15.546325 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c626cfa-b455-432e-a632-af4782dd8386 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:19:15.547399 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c626cfa-b455-432e-a632-af4782dd8386 None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:15.547888 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c626cfa-b455-432e-a632-af4782dd8386 None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:15.548386 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c626cfa-b455-432e-a632-af4782dd8386 None admin] PATH_INFO: `/v3/services` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:15.549286 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c626cfa-b455-432e-a632-af4782dd8386 None admin] RBAC: Adding query filter params (type=network) {{(pid=80119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:19:15.549792 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c626cfa-b455-432e-a632-af4782dd8386 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:19:15.551757 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c626cfa-b455-432e-a632-af4782dd8386 None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:19:15.555998 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71905462-dd94-47a5-9dfe-ac413130faf0 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:19:15.557900 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 414/829] 10.222.0.49 () {56 vars in 1112 bytes} [Wed Jun 15 10:19:15 2022] GET /identity/v3/services?type=network => generated 350 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:19:15.558936 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c626cfa-b455-432e-a632-af4782dd8386 None admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:19:15.563124 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=100 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:19:15.598922 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f832d5f-3a87-4472-8699-a2614770af3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:19:15.599855 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f832d5f-3a87-4472-8699-a2614770af3b None admin] REQUEST_METHOD: `PUT` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:15.600135 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f832d5f-3a87-4472-8699-a2614770af3b None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:15.600406 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f832d5f-3a87-4472-8699-a2614770af3b None admin] PATH_INFO: `/v3/projects/559b3e713f5844b6ae37151d56dd419f/users/103e10244eca444b86b9c51ff314d445/roles/16e76452bb5d4ce68a119ad3c9bde41c` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:15.616594 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f832d5f-3a87-4472-8699-a2614770af3b None admin] RBAC: Authorizing `identity:create_grant(project_id=559b3e713f5844b6ae37151d56dd419f, user_id=103e10244eca444b86b9c51ff314d445, role_id=16e76452bb5d4ce68a119ad3c9bde41c)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:19:15.618121 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f832d5f-3a87-4472-8699-a2614770af3b None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:19:15.643219 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 416/830] 10.222.0.49 () {58 vars in 1439 bytes} [Wed Jun 15 10:19:15 2022] PUT /identity/v3/projects/559b3e713f5844b6ae37151d56dd419f/users/103e10244eca444b86b9c51ff314d445/roles/16e76452bb5d4ce68a119ad3c9bde41c => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:19:15.643219 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f832d5f-3a87-4472-8699-a2614770af3b None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:19:15.709241 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c626cfa-b455-432e-a632-af4782dd8386 None admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:19:15.733586 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:19:15.758511 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ad1af3-6616-426b-81ce-22106ba99ca2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:19:15.759979 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ad1af3-6616-426b-81ce-22106ba99ca2 None admin] REQUEST_METHOD: `PUT` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:15.760640 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ad1af3-6616-426b-81ce-22106ba99ca2 None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:15.761279 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ad1af3-6616-426b-81ce-22106ba99ca2 None admin] PATH_INFO: `/v3/projects/559b3e713f5844b6ae37151d56dd419f/users/3a69f99210444f9696e8b22615fb4a41/roles/8051174e7a7947d5b991ebd793c2e543` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:15.763930 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ad1af3-6616-426b-81ce-22106ba99ca2 None admin] RBAC: Authorizing `identity:create_grant(project_id=559b3e713f5844b6ae37151d56dd419f, user_id=3a69f99210444f9696e8b22615fb4a41, role_id=8051174e7a7947d5b991ebd793c2e543)` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:19:15.765846 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f832d5f-3a87-4472-8699-a2614770af3b None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:19:15.766001 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ad1af3-6616-426b-81ce-22106ba99ca2 None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:19:15.795647 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 415/831] 10.222.0.49 () {58 vars in 1439 bytes} [Wed Jun 15 10:19:15 2022] PUT /identity/v3/projects/559b3e713f5844b6ae37151d56dd419f/users/3a69f99210444f9696e8b22615fb4a41/roles/8051174e7a7947d5b991ebd793c2e543 => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:19:15.811232 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b19c423-8407-4396-a96b-be7d65dfe59f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:19:15.811927 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b19c423-8407-4396-a96b-be7d65dfe59f None admin] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:15.812215 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b19c423-8407-4396-a96b-be7d65dfe59f None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:15.812489 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b19c423-8407-4396-a96b-be7d65dfe59f None admin] PATH_INFO: `/v3/endpoints` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:15.813327 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b19c423-8407-4396-a96b-be7d65dfe59f None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:19:15.815564 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b19c423-8407-4396-a96b-be7d65dfe59f None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:19:15.843895 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 417/832] 10.222.0.49 () {60 vars in 1144 bytes} [Wed Jun 15 10:19:15 2022] POST /identity/v3/endpoints => generated 327 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 10:19:16.187536 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06bb49f-1956-4762-879a-ce6f2b26c02e None None] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:16.187940 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06bb49f-1956-4762-879a-ce6f2b26c02e None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:16.188119 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06bb49f-1956-4762-879a-ce6f2b26c02e None None] PATH_INFO: `/` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:16.189736 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 416/833] 10.222.0.49 () {54 vars in 926 bytes} [Wed Jun 15 10:19:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:19:16.195078 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4400b51c-dd44-4a84-a96e-2b70a74057a4 None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:16.195403 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4400b51c-dd44-4a84-a96e-2b70a74057a4 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:16.195685 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4400b51c-dd44-4a84-a96e-2b70a74057a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:16.215069 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-4400b51c-dd44-4a84-a96e-2b70a74057a4 None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:19:16.216312 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-4400b51c-dd44-4a84-a96e-2b70a74057a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:19:16.417740 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 418/834] 10.222.0.49 () {58 vars in 1023 bytes} [Wed Jun 15 10:19:16 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 15 10:19:16.442571 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06bb49f-1956-4762-879a-ce6f2b26c02e None None] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:19:16.511509 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:19:16.670110 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06bb49f-1956-4762-879a-ce6f2b26c02e None None] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:19:16.706207 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae9f7cc-82b1-4efa-9369-42c022d4d441 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:19:16.706947 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae9f7cc-82b1-4efa-9369-42c022d4d441 None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:16.707261 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae9f7cc-82b1-4efa-9369-42c022d4d441 None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:16.707534 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae9f7cc-82b1-4efa-9369-42c022d4d441 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:16.708184 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae9f7cc-82b1-4efa-9369-42c022d4d441 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:19:16.709548 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae9f7cc-82b1-4efa-9369-42c022d4d441 None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:19:16.721879 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application [None req-bae9f7cc-82b1-4efa-9369-42c022d4d441 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 15 10:19:16.721879 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:19:16.721879 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 15 10:19:16.721879 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 15 10:19:16.721879 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 15 10:19:16.721879 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 15 10:19:16.721879 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 15 10:19:16.721879 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 10:19:16.721879 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 15 10:19:16.721879 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application Jun 15 10:19:16.721879 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 15 10:19:16.721879 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application Jun 15 10:19:16.721879 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:19:16.721879 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:19:16.721879 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:19:16.721879 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:19:16.722822 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:19:16.722822 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:19:16.722822 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:19:16.722822 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:19:16.722822 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:19:16.722822 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:19:16.722822 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:19:16.722822 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 15 10:19:16.722822 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 15 10:19:16.722822 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 15 10:19:16.722822 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 15 10:19:16.722822 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:19:16.722822 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:19:16.722822 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:19:16.722822 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:19:16.722822 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:19:16.723689 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:19:16.723689 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:19:16.723689 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application with Lock( Jun 15 10:19:16.723689 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:19:16.723689 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:19:16.723689 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:19:16.723689 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:19:16.723689 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:19:16.723689 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:19:16.723689 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:19:16.723689 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:19:16.723689 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 15 10:19:16.723689 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 10:19:16.723689 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 15 10:19:16.723689 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application Jun 15 10:19:16.723689 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 417/835] 10.222.0.49 () {56 vars in 1132 bytes} [Wed Jun 15 10:19:16 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 281 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:19:16.729435 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4400b51c-dd44-4a84-a96e-2b70a74057a4 None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:19:16.792760 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:19:16.836566 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4400b51c-dd44-4a84-a96e-2b70a74057a4 None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:19:16.871383 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26b81a2-bda7-428b-a059-1675c541fe96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:19:16.872060 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26b81a2-bda7-428b-a059-1675c541fe96 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:16.872348 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26b81a2-bda7-428b-a059-1675c541fe96 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:16.872624 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26b81a2-bda7-428b-a059-1675c541fe96 None admin] PATH_INFO: `/v3/services` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:16.873332 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c26b81a2-bda7-428b-a059-1675c541fe96 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:19:16.873644 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c26b81a2-bda7-428b-a059-1675c541fe96 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:19:16.874997 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c26b81a2-bda7-428b-a059-1675c541fe96 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:19:16.879897 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 419/836] 10.222.0.49 () {56 vars in 1126 bytes} [Wed Jun 15 10:19:16 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:19:16.885481 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae9f7cc-82b1-4efa-9369-42c022d4d441 None admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:19:16.977001 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1ea40c-8993-4001-8293-138769caab35 None None] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:16.977293 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1ea40c-8993-4001-8293-138769caab35 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:16.977479 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1ea40c-8993-4001-8293-138769caab35 None None] PATH_INFO: `/` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:16.979676 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 420/837] 10.222.0.49 () {54 vars in 926 bytes} [Wed Jun 15 10:19:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 15 10:19:16.986483 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4d041d-aa23-4185-a1a4-f5686ae8cba7 None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:16.987155 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4d041d-aa23-4185-a1a4-f5686ae8cba7 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:16.987646 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4d041d-aa23-4185-a1a4-f5686ae8cba7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:17.002660 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae9f7cc-82b1-4efa-9369-42c022d4d441 None admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:19:17.010523 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-da4d041d-aa23-4185-a1a4-f5686ae8cba7 None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:19:17.012072 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-da4d041d-aa23-4185-a1a4-f5686ae8cba7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:19:17.030051 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=100 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:19:17.040087 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f4419b-53ad-4e4f-a894-6a184db7781c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:19:17.040766 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f4419b-53ad-4e4f-a894-6a184db7781c None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:17.041101 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f4419b-53ad-4e4f-a894-6a184db7781c None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:17.041866 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f4419b-53ad-4e4f-a894-6a184db7781c None admin] PATH_INFO: `/v3/services` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:17.042573 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f4419b-53ad-4e4f-a894-6a184db7781c None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:19:17.042881 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f4419b-53ad-4e4f-a894-6a184db7781c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:19:17.044504 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f4419b-53ad-4e4f-a894-6a184db7781c None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:19:17.049666 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 418/838] 10.222.0.49 () {56 vars in 1126 bytes} [Wed Jun 15 10:19:16 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:19:17.099236 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 421/839] 10.222.0.49 () {58 vars in 1023 bytes} [Wed Jun 15 10:19:16 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 15 10:19:17.118788 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f4419b-53ad-4e4f-a894-6a184db7781c None admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:19:17.234273 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f4419b-53ad-4e4f-a894-6a184db7781c None admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:19:17.269718 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04ec4fb-6b0d-4353-82b6-7f9ca4c2c3cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:19:17.270523 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04ec4fb-6b0d-4353-82b6-7f9ca4c2c3cf None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:17.270813 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04ec4fb-6b0d-4353-82b6-7f9ca4c2c3cf None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:17.271114 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04ec4fb-6b0d-4353-82b6-7f9ca4c2c3cf None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:17.271772 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04ec4fb-6b0d-4353-82b6-7f9ca4c2c3cf None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:19:17.273148 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04ec4fb-6b0d-4353-82b6-7f9ca4c2c3cf None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:19:17.286039 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application [None req-a04ec4fb-6b0d-4353-82b6-7f9ca4c2c3cf None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 15 10:19:17.286039 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:19:17.286039 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 15 10:19:17.286039 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 15 10:19:17.286039 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 15 10:19:17.286039 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 15 10:19:17.286039 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 15 10:19:17.286039 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 10:19:17.286039 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 15 10:19:17.286039 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application Jun 15 10:19:17.286039 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 15 10:19:17.286039 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application Jun 15 10:19:17.286039 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:19:17.286039 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:19:17.286039 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:19:17.286039 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:19:17.286039 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:19:17.287048 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:19:17.287048 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:19:17.287048 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:19:17.287048 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:19:17.287048 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:19:17.287048 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:19:17.287048 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 15 10:19:17.287048 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 15 10:19:17.287048 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 15 10:19:17.287048 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 15 10:19:17.287048 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:19:17.287048 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:19:17.287048 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:19:17.287048 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:19:17.287048 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:19:17.287048 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:19:17.287936 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:19:17.287936 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application with Lock( Jun 15 10:19:17.287936 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:19:17.287936 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:19:17.287936 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:19:17.287936 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:19:17.287936 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:19:17.287936 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:19:17.287936 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:19:17.287936 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:19:17.287936 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 15 10:19:17.287936 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 10:19:17.287936 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 15 10:19:17.287936 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application Jun 15 10:19:17.291728 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 419/840] 10.222.0.49 () {56 vars in 1114 bytes} [Wed Jun 15 10:19:17 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:19:17.294393 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4d041d-aa23-4185-a1a4-f5686ae8cba7 None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:19:17.348544 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:19:17.398231 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4d041d-aa23-4185-a1a4-f5686ae8cba7 None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:19:17.432711 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3cebf7-57e1-4870-8c39-f253e9236371 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:19:17.433378 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3cebf7-57e1-4870-8c39-f253e9236371 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:17.433660 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3cebf7-57e1-4870-8c39-f253e9236371 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:17.433935 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3cebf7-57e1-4870-8c39-f253e9236371 None admin] PATH_INFO: `/v3/services` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:17.434634 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be3cebf7-57e1-4870-8c39-f253e9236371 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:19:17.434938 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be3cebf7-57e1-4870-8c39-f253e9236371 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:19:17.436321 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be3cebf7-57e1-4870-8c39-f253e9236371 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:19:17.441205 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 422/841] 10.222.0.49 () {56 vars in 1114 bytes} [Wed Jun 15 10:19:17 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:19:17.446837 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04ec4fb-6b0d-4353-82b6-7f9ca4c2c3cf None admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:19:17.505011 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=100 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:19:17.554298 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04ec4fb-6b0d-4353-82b6-7f9ca4c2c3cf None admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:19:17.589514 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ec6be2-c1c4-4bbf-a3bb-02b5fad76b6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:19:17.590191 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ec6be2-c1c4-4bbf-a3bb-02b5fad76b6f None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:17.590477 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ec6be2-c1c4-4bbf-a3bb-02b5fad76b6f None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:17.590754 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ec6be2-c1c4-4bbf-a3bb-02b5fad76b6f None admin] PATH_INFO: `/v3/services` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:17.591494 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ec6be2-c1c4-4bbf-a3bb-02b5fad76b6f None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:19:17.591806 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ec6be2-c1c4-4bbf-a3bb-02b5fad76b6f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:19:17.593178 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ec6be2-c1c4-4bbf-a3bb-02b5fad76b6f None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:19:17.598552 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 420/842] 10.222.0.49 () {56 vars in 1114 bytes} [Wed Jun 15 10:19:17 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:19:18.049076 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3d25ef-38d6-4c55-a940-3a123863ceda None None] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:18.049578 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3d25ef-38d6-4c55-a940-3a123863ceda None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:18.049578 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3d25ef-38d6-4c55-a940-3a123863ceda None None] PATH_INFO: `/` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:18.051429 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 423/843] 10.222.0.49 () {54 vars in 926 bytes} [Wed Jun 15 10:19:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:19:18.057211 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ffa3ce-617f-478b-8be8-9cdec68725c7 None None] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:18.057649 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ffa3ce-617f-478b-8be8-9cdec68725c7 None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:18.057745 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ffa3ce-617f-478b-8be8-9cdec68725c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:18.076220 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.auth.core [None req-24ffa3ce-617f-478b-8be8-9cdec68725c7 None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:19:18.077689 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-24ffa3ce-617f-478b-8be8-9cdec68725c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:19:18.148570 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 421/844] 10.222.0.49 () {58 vars in 1023 bytes} [Wed Jun 15 10:19:18 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 10:19:18.172185 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3d25ef-38d6-4c55-a940-3a123863ceda None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:19:18.220624 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033343cf-1ab8-436a-9451-234707f367f3 None None] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:18.220917 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033343cf-1ab8-436a-9451-234707f367f3 None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:18.221199 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033343cf-1ab8-436a-9451-234707f367f3 None None] PATH_INFO: `/` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:18.222805 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 422/845] 10.222.0.49 () {54 vars in 926 bytes} [Wed Jun 15 10:19:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 15 10:19:18.227710 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7c761f-e63f-48ba-abb9-298ea94ddd27 None None] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:18.227993 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7c761f-e63f-48ba-abb9-298ea94ddd27 None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:18.228271 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7c761f-e63f-48ba-abb9-298ea94ddd27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:18.247472 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.auth.core [None req-1d7c761f-e63f-48ba-abb9-298ea94ddd27 None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:19:18.248723 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-1d7c761f-e63f-48ba-abb9-298ea94ddd27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:19:18.281544 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3d25ef-38d6-4c55-a940-3a123863ceda None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:19:18.301290 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:19:18.307655 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=100 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:19:18.323525 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6f1978-ec19-4578-88a7-7787dd942203 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:19:18.324668 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6f1978-ec19-4578-88a7-7787dd942203 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:18.325258 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6f1978-ec19-4578-88a7-7787dd942203 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:18.325602 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6f1978-ec19-4578-88a7-7787dd942203 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:18.333900 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 423/846] 10.222.0.49 () {58 vars in 1023 bytes} [Wed Jun 15 10:19:18 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 15 10:19:18.339194 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a6f1978-ec19-4578-88a7-7787dd942203 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 10:19:18.347207 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application [None req-7a6f1978-ec19-4578-88a7-7787dd942203 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 15 10:19:18.347207 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:19:18.347207 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:19:18.347207 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:19:18.347207 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:19:18.347207 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:19:18.347207 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:19:18.347207 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:19:18.347207 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:19:18.347207 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:19:18.347207 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:19:18.347207 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:19:18.347207 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 10:19:18.347207 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 10:19:18.347207 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 10:19:18.347207 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 10:19:18.348156 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 10:19:18.348156 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 10:19:18.348156 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 10:19:18.348156 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 10:19:18.348156 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:19:18.348156 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:19:18.348156 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:19:18.348156 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:19:18.348156 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:19:18.348156 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:19:18.348156 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:19:18.348156 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application with Lock( Jun 15 10:19:18.348156 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:19:18.348156 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:19:18.348156 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:19:18.348156 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:19:18.349228 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:19:18.349228 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:19:18.349228 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:19:18.349228 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:19:18.349228 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 10:19:18.349228 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 10:19:18.349228 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 10:19:18.349228 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 10:19:18.349228 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 10:19:18.349228 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 10:19:18.349228 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 15 10:19:18.349228 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Jun 15 10:19:18.349228 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 424/847] 10.222.0.49 () {56 vars in 1102 bytes} [Wed Jun 15 10:19:18 2022] GET /identity/v3/roles/service => generated 85 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:19:18.354699 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7c761f-e63f-48ba-abb9-298ea94ddd27 None None] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:19:18.360843 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6f1978-ec19-4578-88a7-7787dd942203 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:19:18.459376 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7c761f-e63f-48ba-abb9-298ea94ddd27 None None] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:19:18.494630 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2575294c-c6c1-445c-8f61-53a19bc05136 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:19:18.495823 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2575294c-c6c1-445c-8f61-53a19bc05136 None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:18.496105 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2575294c-c6c1-445c-8f61-53a19bc05136 None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:18.496376 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2575294c-c6c1-445c-8f61-53a19bc05136 None admin] PATH_INFO: `/v3/roles` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:18.497119 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2575294c-c6c1-445c-8f61-53a19bc05136 None admin] RBAC: Adding query filter params (name=service) {{(pid=80119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:19:18.497420 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2575294c-c6c1-445c-8f61-53a19bc05136 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:19:18.497614 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6f1978-ec19-4578-88a7-7787dd942203 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:19:18.498785 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2575294c-c6c1-445c-8f61-53a19bc05136 None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:19:18.512074 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 424/848] 10.222.0.49 () {56 vars in 1103 bytes} [Wed Jun 15 10:19:18 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:19:18.513122 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2575294c-c6c1-445c-8f61-53a19bc05136 None admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:19:18.539731 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0cbcd95-9869-48b8-b386-62982b4d69e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:19:18.540978 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cbcd95-9869-48b8-b386-62982b4d69e2 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:18.542022 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cbcd95-9869-48b8-b386-62982b4d69e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:18.542349 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cbcd95-9869-48b8-b386-62982b4d69e2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:18.559955 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0cbcd95-9869-48b8-b386-62982b4d69e2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 10:19:18.560818 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=100 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:19:18.572112 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application [None req-e0cbcd95-9869-48b8-b386-62982b4d69e2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 10:19:18.572112 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:19:18.572112 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:19:18.572112 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:19:18.572112 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:19:18.572112 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:19:18.572112 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:19:18.572112 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:19:18.572112 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:19:18.572112 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:19:18.572112 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:19:18.572112 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:19:18.572112 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 10:19:18.572112 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 10:19:18.572112 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 10:19:18.572112 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 10:19:18.573518 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 10:19:18.573518 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 10:19:18.573518 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 10:19:18.573518 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 10:19:18.573518 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:19:18.573518 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:19:18.573518 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:19:18.573518 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:19:18.573518 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:19:18.573518 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:19:18.573518 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:19:18.573518 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application with Lock( Jun 15 10:19:18.573518 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:19:18.573518 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:19:18.573518 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:19:18.573518 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:19:18.575258 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:19:18.575258 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:19:18.575258 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:19:18.575258 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:19:18.575258 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 10:19:18.575258 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 10:19:18.575258 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 10:19:18.575258 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 10:19:18.575258 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 10:19:18.575258 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 10:19:18.575258 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 10:19:18.575258 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Jun 15 10:19:18.583112 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 425/849] 10.222.0.49 () {56 vars in 1096 bytes} [Wed Jun 15 10:19:18 2022] GET /identity/v3/roles/admin => generated 83 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:19:18.583112 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0cbcd95-9869-48b8-b386-62982b4d69e2 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:19:18.627238 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2575294c-c6c1-445c-8f61-53a19bc05136 None admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:19:18.634099 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:19:18.668279 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124c8bc5-36e3-4091-898d-fad746e86e49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:19:18.669221 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124c8bc5-36e3-4091-898d-fad746e86e49 None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:18.669600 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124c8bc5-36e3-4091-898d-fad746e86e49 None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:18.669994 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124c8bc5-36e3-4091-898d-fad746e86e49 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:18.681105 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-124c8bc5-36e3-4091-898d-fad746e86e49 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:19:18.682706 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-124c8bc5-36e3-4091-898d-fad746e86e49 None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:19:18.700488 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application [None req-124c8bc5-36e3-4091-898d-fad746e86e49 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 10:19:18.700488 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:19:18.700488 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:19:18.700488 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:19:18.700488 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:19:18.700488 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:19:18.700488 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:19:18.700488 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:19:18.700488 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:19:18.700488 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:19:18.700488 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:19:18.700488 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:19:18.700488 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 10:19:18.700488 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 10:19:18.700488 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 10:19:18.701624 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 10:19:18.701624 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:19:18.701624 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:19:18.701624 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:19:18.701624 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:19:18.701624 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:19:18.701624 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:19:18.701624 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:19:18.701624 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application with Lock( Jun 15 10:19:18.701624 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:19:18.701624 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:19:18.701624 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:19:18.701624 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:19:18.701624 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:19:18.701624 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:19:18.701624 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:19:18.701624 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:19:18.702761 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 10:19:18.702761 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 10:19:18.702761 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 10:19:18.702761 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application Jun 15 10:19:18.707332 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 425/850] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Jun 15 10:19:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 10:19:18.709847 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124c8bc5-36e3-4091-898d-fad746e86e49 None admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:19:18.711628 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0cbcd95-9869-48b8-b386-62982b4d69e2 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:19:18.765196 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784e55ad-053e-4a92-a788-ebdccf639454 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:19:18.765875 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784e55ad-053e-4a92-a788-ebdccf639454 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:18.766155 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784e55ad-053e-4a92-a788-ebdccf639454 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:18.766426 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784e55ad-053e-4a92-a788-ebdccf639454 None admin] PATH_INFO: `/v3/roles` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:18.767212 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-784e55ad-053e-4a92-a788-ebdccf639454 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:19:18.767519 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-784e55ad-053e-4a92-a788-ebdccf639454 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:19:18.769409 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-784e55ad-053e-4a92-a788-ebdccf639454 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:19:18.783447 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 426/851] 10.222.0.49 () {56 vars in 1099 bytes} [Wed Jun 15 10:19:18 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:19:18.784361 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784e55ad-053e-4a92-a788-ebdccf639454 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:19:18.823883 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=100 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:19:18.835611 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124c8bc5-36e3-4091-898d-fad746e86e49 None admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:19:18.880223 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fed960e-2dc7-431a-a10e-2a25664e9dac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:19:18.881405 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fed960e-2dc7-431a-a10e-2a25664e9dac None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:18.881971 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fed960e-2dc7-431a-a10e-2a25664e9dac None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:18.882968 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fed960e-2dc7-431a-a10e-2a25664e9dac None admin] PATH_INFO: `/v3/domains` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:18.884058 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fed960e-2dc7-431a-a10e-2a25664e9dac None admin] RBAC: Adding query filter params (name=Default) {{(pid=80119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:19:18.885448 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fed960e-2dc7-431a-a10e-2a25664e9dac None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:19:18.887526 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fed960e-2dc7-431a-a10e-2a25664e9dac None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:19:18.903829 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 426/852] 10.222.0.49 () {56 vars in 1109 bytes} [Wed Jun 15 10:19:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:19:18.909563 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fed960e-2dc7-431a-a10e-2a25664e9dac None admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:19:18.912919 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:19:18.917935 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784e55ad-053e-4a92-a788-ebdccf639454 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:19:18.954572 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d87839-97be-4243-91cb-7c5b36018846 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:19:18.955440 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d87839-97be-4243-91cb-7c5b36018846 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:18.955723 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d87839-97be-4243-91cb-7c5b36018846 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:18.955992 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d87839-97be-4243-91cb-7c5b36018846 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:18.968128 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d87839-97be-4243-91cb-7c5b36018846 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:19:18.969723 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d87839-97be-4243-91cb-7c5b36018846 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:19:18.988463 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application [None req-52d87839-97be-4243-91cb-7c5b36018846 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 10:19:18.988463 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:19:18.988463 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:19:18.988463 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:19:18.988463 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:19:18.988463 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:19:18.988463 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:19:18.988463 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:19:18.988463 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:19:18.988463 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:19:18.988463 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:19:18.988463 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:19:18.988463 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 10:19:18.988463 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 10:19:18.988463 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 10:19:18.989419 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 10:19:18.989419 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:19:18.989419 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:19:18.989419 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:19:18.989419 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:19:18.989419 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:19:18.989419 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:19:18.989419 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:19:18.989419 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application with Lock( Jun 15 10:19:18.989419 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:19:18.989419 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:19:18.989419 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:19:18.989419 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:19:18.989419 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:19:18.989419 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:19:18.989419 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:19:18.989419 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:19:18.990417 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 10:19:18.990417 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 10:19:18.990417 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 10:19:18.990417 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Jun 15 10:19:18.994794 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 427/853] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Jun 15 10:19:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:19:18.995975 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d87839-97be-4243-91cb-7c5b36018846 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:19:19.039533 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fed960e-2dc7-431a-a10e-2a25664e9dac None admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:19:19.085357 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80abe8ec-49ef-46b2-b8fa-a26ea8a3cce5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:19:19.087555 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80abe8ec-49ef-46b2-b8fa-a26ea8a3cce5 None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:19.088538 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80abe8ec-49ef-46b2-b8fa-a26ea8a3cce5 None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:19.088964 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80abe8ec-49ef-46b2-b8fa-a26ea8a3cce5 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:19.098926 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80abe8ec-49ef-46b2-b8fa-a26ea8a3cce5 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:19:19.101525 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80abe8ec-49ef-46b2-b8fa-a26ea8a3cce5 None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:19:19.106660 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d87839-97be-4243-91cb-7c5b36018846 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:19:19.122381 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application [None req-80abe8ec-49ef-46b2-b8fa-a26ea8a3cce5 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 15 10:19:19.122381 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:19:19.122381 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:19:19.122381 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:19:19.122381 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:19:19.122381 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:19:19.122381 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:19:19.122381 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:19:19.122381 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:19:19.122381 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:19:19.122381 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:19:19.122381 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:19:19.122381 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 15 10:19:19.122381 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 15 10:19:19.122381 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 15 10:19:19.122381 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 15 10:19:19.123588 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:19:19.123588 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:19:19.123588 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 15 10:19:19.123588 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 10:19:19.123588 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 15 10:19:19.123588 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 10:19:19.123588 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:19:19.123588 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:19:19.123588 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:19:19.123588 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:19:19.123588 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:19:19.123588 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application with Lock( Jun 15 10:19:19.123588 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:19:19.123588 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:19:19.123588 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:19:19.123588 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:19:19.123588 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:19:19.123588 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:19:19.124798 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:19:19.124798 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:19:19.124798 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 15 10:19:19.124798 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 15 10:19:19.124798 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 15 10:19:19.124798 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 15 10:19:19.124798 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 15 10:19:19.124798 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 15 10:19:19.124798 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 15 10:19:19.124798 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application Jun 15 10:19:19.132600 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 427/854] 10.222.0.49 () {56 vars in 1117 bytes} [Wed Jun 15 10:19:18 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:19:19.134045 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80abe8ec-49ef-46b2-b8fa-a26ea8a3cce5 None admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:19:19.148252 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f10132-2b3a-4166-9aa4-741968cb69d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:19:19.149094 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f10132-2b3a-4166-9aa4-741968cb69d2 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:19.149483 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f10132-2b3a-4166-9aa4-741968cb69d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:19.149826 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f10132-2b3a-4166-9aa4-741968cb69d2 None admin] PATH_INFO: `/v3/domains` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:19.150726 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f10132-2b3a-4166-9aa4-741968cb69d2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:19:19.151831 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f10132-2b3a-4166-9aa4-741968cb69d2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:19:19.153232 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f10132-2b3a-4166-9aa4-741968cb69d2 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:19:19.154727 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=100 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:19:19.175863 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 428/855] 10.222.0.49 () {56 vars in 1109 bytes} [Wed Jun 15 10:19:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:19:19.176859 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f10132-2b3a-4166-9aa4-741968cb69d2 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:19:19.192927 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:19:19.258711 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80abe8ec-49ef-46b2-b8fa-a26ea8a3cce5 None admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:19:19.301725 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f10132-2b3a-4166-9aa4-741968cb69d2 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:19:19.302442 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b6eb56-7f3b-435f-bcd4-ead729005d0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:19:19.303411 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b6eb56-7f3b-435f-bcd4-ead729005d0c None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:19.304109 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b6eb56-7f3b-435f-bcd4-ead729005d0c None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:19.304570 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b6eb56-7f3b-435f-bcd4-ead729005d0c None admin] PATH_INFO: `/v3/users` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:19.307146 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b6eb56-7f3b-435f-bcd4-ead729005d0c None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:19:19.307674 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b6eb56-7f3b-435f-bcd4-ead729005d0c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:19:19.309377 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b6eb56-7f3b-435f-bcd4-ead729005d0c None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:19:19.337723 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 428/856] 10.222.0.49 () {56 vars in 1149 bytes} [Wed Jun 15 10:19:19 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:19:19.338747 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b6eb56-7f3b-435f-bcd4-ead729005d0c None admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:19:19.346932 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab41439-288f-4a17-95a8-6b9503640632 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:19:19.347775 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab41439-288f-4a17-95a8-6b9503640632 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:19.348106 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab41439-288f-4a17-95a8-6b9503640632 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:19.348379 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab41439-288f-4a17-95a8-6b9503640632 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:19.356737 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab41439-288f-4a17-95a8-6b9503640632 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:19:19.358256 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab41439-288f-4a17-95a8-6b9503640632 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:19:19.373126 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application [None req-fab41439-288f-4a17-95a8-6b9503640632 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 15 10:19:19.373126 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:19:19.373126 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:19:19.373126 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:19:19.373126 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:19:19.373126 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:19:19.373126 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:19:19.373126 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:19:19.373126 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:19:19.373126 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:19:19.373126 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:19:19.373126 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:19:19.373126 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 15 10:19:19.373126 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 15 10:19:19.373126 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 15 10:19:19.373126 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 15 10:19:19.374090 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:19:19.374090 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:19:19.374090 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 15 10:19:19.374090 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 10:19:19.374090 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 15 10:19:19.374090 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 10:19:19.374090 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:19:19.374090 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:19:19.374090 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:19:19.374090 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:19:19.374090 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:19:19.374090 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application with Lock( Jun 15 10:19:19.374090 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:19:19.374090 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:19:19.374090 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:19:19.374090 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:19:19.374090 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:19:19.374090 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:19:19.375053 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:19:19.375053 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:19:19.375053 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 15 10:19:19.375053 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 15 10:19:19.375053 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 15 10:19:19.375053 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 15 10:19:19.375053 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 15 10:19:19.375053 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 15 10:19:19.375053 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 15 10:19:19.375053 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Jun 15 10:19:19.375053 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 429/857] 10.222.0.49 () {56 vars in 1096 bytes} [Wed Jun 15 10:19:19 2022] GET /identity/v3/users/swift => generated 83 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:19:19.375675 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451a243b-4d26-4f3e-a17e-110f8f727d5c None None] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:19.375675 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451a243b-4d26-4f3e-a17e-110f8f727d5c None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:19.376460 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451a243b-4d26-4f3e-a17e-110f8f727d5c None None] PATH_INFO: `/` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:19.377393 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 430/858] 10.222.0.49 () {54 vars in 926 bytes} [Wed Jun 15 10:19:19 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:19:19.381580 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451a243b-4d26-4f3e-a17e-110f8f727d5c None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:19:19.421181 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=100 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:19:19.455684 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b6eb56-7f3b-435f-bcd4-ead729005d0c None admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:19:19.458623 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:19:19.490317 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451a243b-4d26-4f3e-a17e-110f8f727d5c None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:19:19.492135 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e4a773-0f81-41e6-99e1-5e2ef4f2f313 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:19:19.493190 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e4a773-0f81-41e6-99e1-5e2ef4f2f313 None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:19.493190 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e4a773-0f81-41e6-99e1-5e2ef4f2f313 None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:19.493422 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e4a773-0f81-41e6-99e1-5e2ef4f2f313 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:19.530710 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e832e457-70bc-4ef3-879f-cb2d7a394b1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:19:19.531415 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e832e457-70bc-4ef3-879f-cb2d7a394b1f None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:19.531691 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e832e457-70bc-4ef3-879f-cb2d7a394b1f None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:19.531959 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e832e457-70bc-4ef3-879f-cb2d7a394b1f None admin] PATH_INFO: `/v3/users` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:19.532791 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e832e457-70bc-4ef3-879f-cb2d7a394b1f None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:19:19.533096 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e832e457-70bc-4ef3-879f-cb2d7a394b1f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:19:19.534501 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e832e457-70bc-4ef3-879f-cb2d7a394b1f None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:19:19.554244 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 431/859] 10.222.0.49 () {56 vars in 1135 bytes} [Wed Jun 15 10:19:19 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:19:19.557184 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10a122e-3b70-4b5f-8e2e-7893afb9cbe8 None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:19.557705 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10a122e-3b70-4b5f-8e2e-7893afb9cbe8 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:19.557990 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10a122e-3b70-4b5f-8e2e-7893afb9cbe8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:19.566443 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e4a773-0f81-41e6-99e1-5e2ef4f2f313 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:19:19.568171 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e4a773-0f81-41e6-99e1-5e2ef4f2f313 None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:19:19.575817 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-a10a122e-3b70-4b5f-8e2e-7893afb9cbe8 None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:19:19.577010 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-a10a122e-3b70-4b5f-8e2e-7893afb9cbe8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:19:19.645382 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=100 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:19:19.648489 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 432/860] 10.222.0.49 () {58 vars in 1023 bytes} [Wed Jun 15 10:19:19 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 10:19:19.649393 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10a122e-3b70-4b5f-8e2e-7893afb9cbe8 None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:19:19.687440 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 429/861] 10.222.0.49 () {58 vars in 1282 bytes} [Wed Jun 15 10:19:19 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 15 10:19:19.688302 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e4a773-0f81-41e6-99e1-5e2ef4f2f313 None admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:19:19.718488 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:19:19.764340 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10a122e-3b70-4b5f-8e2e-7893afb9cbe8 None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:19:19.799135 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e4a773-0f81-41e6-99e1-5e2ef4f2f313 None admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:19:19.805497 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0552a8f-7f83-4099-8433-548f4ad57e76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:19:19.806155 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0552a8f-7f83-4099-8433-548f4ad57e76 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:19.806440 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0552a8f-7f83-4099-8433-548f4ad57e76 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:19.807346 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0552a8f-7f83-4099-8433-548f4ad57e76 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:19.833485 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49be8d09-9ec8-4094-99e4-630eb4084152 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:19:19.834142 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49be8d09-9ec8-4094-99e4-630eb4084152 None admin] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:19.834420 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49be8d09-9ec8-4094-99e4-630eb4084152 None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:19.834694 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49be8d09-9ec8-4094-99e4-630eb4084152 None admin] PATH_INFO: `/v3/services` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:19.835427 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49be8d09-9ec8-4094-99e4-630eb4084152 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:19:19.836788 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49be8d09-9ec8-4094-99e4-630eb4084152 None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:19:19.849151 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 430/862] 10.222.0.49 () {60 vars in 1141 bytes} [Wed Jun 15 10:19:19 2022] POST /identity/v3/services => generated 269 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 10:19:19.849900 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49be8d09-9ec8-4094-99e4-630eb4084152 None admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:19:19.880436 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0552a8f-7f83-4099-8433-548f4ad57e76 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:19:19.882110 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0552a8f-7f83-4099-8433-548f4ad57e76 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:19:19.892804 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:19:19.954106 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:19:19.977058 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49be8d09-9ec8-4094-99e4-630eb4084152 None admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:19:20.007991 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 433/863] 10.222.0.49 () {58 vars in 1282 bytes} [Wed Jun 15 10:19:19 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 15 10:19:20.009690 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d22a26-7301-4382-a5e2-cd0fc83174c2 None None] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:20.010137 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d22a26-7301-4382-a5e2-cd0fc83174c2 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:20.010412 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d22a26-7301-4382-a5e2-cd0fc83174c2 None None] PATH_INFO: `/` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:20.012073 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 434/864] 10.222.0.49 () {54 vars in 926 bytes} [Wed Jun 15 10:19:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:19:20.012779 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d22a26-7301-4382-a5e2-cd0fc83174c2 None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:19:20.013186 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405f12c9-4351-4b03-a3b0-f6fdc8b1b133 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:19:20.013987 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405f12c9-4351-4b03-a3b0-f6fdc8b1b133 None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:20.014271 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405f12c9-4351-4b03-a3b0-f6fdc8b1b133 None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:20.014548 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405f12c9-4351-4b03-a3b0-f6fdc8b1b133 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:20.029764 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405f12c9-4351-4b03-a3b0-f6fdc8b1b133 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:19:20.031271 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405f12c9-4351-4b03-a3b0-f6fdc8b1b133 None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:19:20.052376 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application [None req-405f12c9-4351-4b03-a3b0-f6fdc8b1b133 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 10:19:20.052376 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:19:20.052376 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:19:20.052376 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:19:20.052376 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:19:20.052376 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:19:20.052376 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:19:20.052376 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:19:20.052376 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:19:20.052376 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:19:20.052376 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:19:20.052376 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:19:20.052376 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 10:19:20.052376 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 10:19:20.052376 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 10:19:20.053355 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 10:19:20.053355 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:19:20.053355 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:19:20.053355 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:19:20.053355 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:19:20.053355 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:19:20.053355 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:19:20.053355 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:19:20.053355 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application with Lock( Jun 15 10:19:20.053355 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:19:20.053355 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:19:20.053355 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:19:20.053355 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:19:20.053355 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:19:20.053355 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:19:20.053355 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:19:20.053355 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:19:20.054391 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 10:19:20.054391 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 10:19:20.054391 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 10:19:20.054391 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application Jun 15 10:19:20.054391 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 431/865] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Jun 15 10:19:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:19:20.055473 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae5bead-e02b-45e1-a1a9-fc90c27202eb None None] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:20.055758 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae5bead-e02b-45e1-a1a9-fc90c27202eb None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:20.056036 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae5bead-e02b-45e1-a1a9-fc90c27202eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:20.072544 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.auth.core [None req-eae5bead-e02b-45e1-a1a9-fc90c27202eb None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:19:20.073736 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-eae5bead-e02b-45e1-a1a9-fc90c27202eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:19:20.126460 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d22a26-7301-4382-a5e2-cd0fc83174c2 None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:19:20.144931 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 432/866] 10.222.0.49 () {58 vars in 1023 bytes} [Wed Jun 15 10:19:20 2022] POST /identity/v3/auth/tokens => generated 1261 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 10:19:20.145785 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae5bead-e02b-45e1-a1a9-fc90c27202eb None None] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:19:20.159069 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=100 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:19:20.168729 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917a2845-bc36-4dcd-b270-be9e0da7978d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:19:20.169669 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917a2845-bc36-4dcd-b270-be9e0da7978d None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:20.170076 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917a2845-bc36-4dcd-b270-be9e0da7978d None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:20.170721 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917a2845-bc36-4dcd-b270-be9e0da7978d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:20.181578 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917a2845-bc36-4dcd-b270-be9e0da7978d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:19:20.183139 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917a2845-bc36-4dcd-b270-be9e0da7978d None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:19:20.201888 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application [None req-917a2845-bc36-4dcd-b270-be9e0da7978d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 10:19:20.201888 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:19:20.201888 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:19:20.201888 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:19:20.201888 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:19:20.201888 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:19:20.201888 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:19:20.201888 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:19:20.201888 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:19:20.201888 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:19:20.201888 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:19:20.201888 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:19:20.201888 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 10:19:20.201888 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 10:19:20.201888 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 10:19:20.203019 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 10:19:20.203019 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:19:20.203019 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:19:20.203019 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:19:20.203019 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:19:20.203019 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:19:20.203019 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:19:20.203019 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:19:20.203019 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application with Lock( Jun 15 10:19:20.203019 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:19:20.203019 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:19:20.203019 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:19:20.203019 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:19:20.203019 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:19:20.203019 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:19:20.203019 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:19:20.203019 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:19:20.204238 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 10:19:20.204238 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 10:19:20.204238 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 10:19:20.204238 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Jun 15 10:19:20.210178 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 435/867] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Jun 15 10:19:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:19:20.210178 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917a2845-bc36-4dcd-b270-be9e0da7978d None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:19:20.222237 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:19:20.255581 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae5bead-e02b-45e1-a1a9-fc90c27202eb None None] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:19:20.291902 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b8e335-09d4-4e4a-b6a2-5d0e25d2c727 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:19:20.292659 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b8e335-09d4-4e4a-b6a2-5d0e25d2c727 None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:20.292944 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b8e335-09d4-4e4a-b6a2-5d0e25d2c727 None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:20.293218 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b8e335-09d4-4e4a-b6a2-5d0e25d2c727 None admin] PATH_INFO: `/v3/domains` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:20.293930 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b8e335-09d4-4e4a-b6a2-5d0e25d2c727 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:19:20.294232 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b8e335-09d4-4e4a-b6a2-5d0e25d2c727 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:19:20.295650 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b8e335-09d4-4e4a-b6a2-5d0e25d2c727 None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:19:20.307456 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 433/868] 10.222.0.49 () {56 vars in 1109 bytes} [Wed Jun 15 10:19:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:19:20.308256 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b8e335-09d4-4e4a-b6a2-5d0e25d2c727 None admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:19:20.315574 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917a2845-bc36-4dcd-b270-be9e0da7978d None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:19:20.355128 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e996011f-d5e8-4cd6-b43d-13d9acfc36c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:19:20.355828 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e996011f-d5e8-4cd6-b43d-13d9acfc36c5 None admin] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:20.356113 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e996011f-d5e8-4cd6-b43d-13d9acfc36c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:20.356389 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e996011f-d5e8-4cd6-b43d-13d9acfc36c5 None admin] PATH_INFO: `/v3/services` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:20.357109 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e996011f-d5e8-4cd6-b43d-13d9acfc36c5 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:19:20.358468 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e996011f-d5e8-4cd6-b43d-13d9acfc36c5 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:19:20.373240 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 436/869] 10.222.0.49 () {60 vars in 1141 bytes} [Wed Jun 15 10:19:20 2022] POST /identity/v3/services => generated 251 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 10:19:20.373240 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e996011f-d5e8-4cd6-b43d-13d9acfc36c5 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:19:20.382963 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=100 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:19:20.421946 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b8e335-09d4-4e4a-b6a2-5d0e25d2c727 None admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:19:20.460381 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:19:20.469536 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b320ba-a329-4bcd-8e9d-c3400f9ad1f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:19:20.470665 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b320ba-a329-4bcd-8e9d-c3400f9ad1f9 None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:20.471189 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b320ba-a329-4bcd-8e9d-c3400f9ad1f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:20.471664 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b320ba-a329-4bcd-8e9d-c3400f9ad1f9 None admin] PATH_INFO: `/v3/domains` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:20.472704 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b320ba-a329-4bcd-8e9d-c3400f9ad1f9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:19:20.473309 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b320ba-a329-4bcd-8e9d-c3400f9ad1f9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:19:20.475175 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b320ba-a329-4bcd-8e9d-c3400f9ad1f9 None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:19:20.489246 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 434/870] 10.222.0.49 () {56 vars in 1109 bytes} [Wed Jun 15 10:19:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:19:20.499134 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b320ba-a329-4bcd-8e9d-c3400f9ad1f9 None admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:19:20.501933 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e996011f-d5e8-4cd6-b43d-13d9acfc36c5 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:19:20.547770 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da269e72-46e1-4a4a-aadb-7f69565715b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:19:20.548582 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da269e72-46e1-4a4a-aadb-7f69565715b9 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:20.548865 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da269e72-46e1-4a4a-aadb-7f69565715b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:20.549136 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da269e72-46e1-4a4a-aadb-7f69565715b9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:20.557837 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da269e72-46e1-4a4a-aadb-7f69565715b9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:19:20.559732 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da269e72-46e1-4a4a-aadb-7f69565715b9 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:19:20.576837 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application [None req-da269e72-46e1-4a4a-aadb-7f69565715b9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 10:19:20.576837 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:19:20.576837 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:19:20.576837 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:19:20.576837 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:19:20.576837 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:19:20.576837 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:19:20.576837 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:19:20.576837 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:19:20.576837 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:19:20.576837 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:19:20.576837 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:19:20.576837 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 15 10:19:20.576837 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 15 10:19:20.576837 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 15 10:19:20.578007 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 15 10:19:20.578007 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:19:20.578007 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:19:20.578007 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:19:20.578007 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:19:20.578007 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:19:20.578007 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:19:20.578007 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:19:20.578007 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application with Lock( Jun 15 10:19:20.578007 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:19:20.578007 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:19:20.578007 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:19:20.578007 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:19:20.578007 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:19:20.578007 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:19:20.578007 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:19:20.578007 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:19:20.579174 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 15 10:19:20.579174 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 15 10:19:20.579174 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 15 10:19:20.579174 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 15 10:19:20.579174 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 15 10:19:20.579174 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 15 10:19:20.579174 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 10:19:20.579174 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Jun 15 10:19:20.579174 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 437/871] 10.222.0.49 () {56 vars in 1111 bytes} [Wed Jun 15 10:19:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:19:20.584574 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da269e72-46e1-4a4a-aadb-7f69565715b9 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:19:20.628268 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b320ba-a329-4bcd-8e9d-c3400f9ad1f9 None admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:19:20.661325 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=100 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:19:20.673188 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd9c688-082b-40fe-9c53-15d4eb50f476 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:19:20.674070 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd9c688-082b-40fe-9c53-15d4eb50f476 None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:20.674418 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd9c688-082b-40fe-9c53-15d4eb50f476 None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:20.675007 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd9c688-082b-40fe-9c53-15d4eb50f476 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:20.682557 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd9c688-082b-40fe-9c53-15d4eb50f476 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:19:20.684117 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd9c688-082b-40fe-9c53-15d4eb50f476 None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:19:20.699212 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application [None req-fcd9c688-082b-40fe-9c53-15d4eb50f476 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 10:19:20.699212 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:19:20.699212 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:19:20.699212 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:19:20.699212 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:19:20.699212 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:19:20.699212 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:19:20.699212 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:19:20.699212 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:19:20.699212 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:19:20.699212 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:19:20.699212 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:19:20.699212 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 15 10:19:20.699212 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 15 10:19:20.699212 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 15 10:19:20.700108 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 15 10:19:20.700108 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:19:20.700108 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:19:20.700108 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:19:20.700108 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:19:20.700108 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:19:20.700108 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:19:20.700108 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:19:20.700108 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application with Lock( Jun 15 10:19:20.700108 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:19:20.700108 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:19:20.700108 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:19:20.700108 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:19:20.700108 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:19:20.700108 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:19:20.700108 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:19:20.700108 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:19:20.701007 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 15 10:19:20.701007 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 15 10:19:20.701007 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 15 10:19:20.701007 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 15 10:19:20.701007 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 15 10:19:20.701007 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 15 10:19:20.701007 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 10:19:20.701007 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application Jun 15 10:19:20.701007 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 435/872] 10.222.0.49 () {56 vars in 1111 bytes} [Wed Jun 15 10:19:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:19:20.707753 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd9c688-082b-40fe-9c53-15d4eb50f476 None admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:19:20.719950 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da269e72-46e1-4a4a-aadb-7f69565715b9 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:19:20.760543 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:19:20.768133 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a74004-eddb-4afc-9a11-80c01f4ce3c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:19:20.768803 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a74004-eddb-4afc-9a11-80c01f4ce3c6 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:20.769343 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a74004-eddb-4afc-9a11-80c01f4ce3c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:20.771513 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a74004-eddb-4afc-9a11-80c01f4ce3c6 None admin] PATH_INFO: `/v3/projects` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:20.772364 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a74004-eddb-4afc-9a11-80c01f4ce3c6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:19:20.772682 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a74004-eddb-4afc-9a11-80c01f4ce3c6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:19:20.774109 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a74004-eddb-4afc-9a11-80c01f4ce3c6 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:19:20.787936 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 438/873] 10.222.0.49 () {56 vars in 1148 bytes} [Wed Jun 15 10:19:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 10:19:20.795565 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a74004-eddb-4afc-9a11-80c01f4ce3c6 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:19:20.827808 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd9c688-082b-40fe-9c53-15d4eb50f476 None admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:19:20.862572 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9b33ae-28b6-4b6a-9cc1-b4e0a952c693 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:19:20.863527 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9b33ae-28b6-4b6a-9cc1-b4e0a952c693 None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:20.863954 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9b33ae-28b6-4b6a-9cc1-b4e0a952c693 None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:20.864408 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9b33ae-28b6-4b6a-9cc1-b4e0a952c693 None admin] PATH_INFO: `/v3/projects` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:20.865368 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d9b33ae-28b6-4b6a-9cc1-b4e0a952c693 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:19:20.865845 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d9b33ae-28b6-4b6a-9cc1-b4e0a952c693 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:19:20.867440 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d9b33ae-28b6-4b6a-9cc1-b4e0a952c693 None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:19:20.886766 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 436/874] 10.222.0.49 () {56 vars in 1148 bytes} [Wed Jun 15 10:19:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 10:19:20.888246 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9b33ae-28b6-4b6a-9cc1-b4e0a952c693 None admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:19:20.918308 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a74004-eddb-4afc-9a11-80c01f4ce3c6 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:19:20.935272 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=100 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:19:20.963508 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d24031-f4df-4d1e-9285-b1eb68686513 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:19:20.965097 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d24031-f4df-4d1e-9285-b1eb68686513 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:20.965387 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d24031-f4df-4d1e-9285-b1eb68686513 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:20.965667 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d24031-f4df-4d1e-9285-b1eb68686513 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:20.966526 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d24031-f4df-4d1e-9285-b1eb68686513 None admin] RBAC: Adding query filter params (role.id=16e76452bb5d4ce68a119ad3c9bde41c, scope.project.id=559b3e713f5844b6ae37151d56dd419f, user.id=103e10244eca444b86b9c51ff314d445) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:19:20.966840 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d24031-f4df-4d1e-9285-b1eb68686513 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:19:20.968318 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d24031-f4df-4d1e-9285-b1eb68686513 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:19:20.976649 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 439/875] 10.222.0.49 () {56 vars in 1374 bytes} [Wed Jun 15 10:19:20 2022] GET /identity/v3/role_assignments?user.id=103e10244eca444b86b9c51ff314d445&scope.project.id=559b3e713f5844b6ae37151d56dd419f&role.id=16e76452bb5d4ce68a119ad3c9bde41c => generated 608 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:19:21.004801 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9b33ae-28b6-4b6a-9cc1-b4e0a952c693 None admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:19:21.045808 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a41ef7-37c6-473e-9b56-5616689a0602 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:19:21.046777 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a41ef7-37c6-473e-9b56-5616689a0602 None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:21.047255 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a41ef7-37c6-473e-9b56-5616689a0602 None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:21.047673 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a41ef7-37c6-473e-9b56-5616689a0602 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:21.048647 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a41ef7-37c6-473e-9b56-5616689a0602 None admin] RBAC: Adding query filter params (role.id=8051174e7a7947d5b991ebd793c2e543, scope.project.id=559b3e713f5844b6ae37151d56dd419f, user.id=3a69f99210444f9696e8b22615fb4a41) {{(pid=80119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:19:21.049124 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a41ef7-37c6-473e-9b56-5616689a0602 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:19:21.051007 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a41ef7-37c6-473e-9b56-5616689a0602 None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:19:21.063228 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 437/876] 10.222.0.49 () {56 vars in 1374 bytes} [Wed Jun 15 10:19:20 2022] GET /identity/v3/role_assignments?user.id=3a69f99210444f9696e8b22615fb4a41&scope.project.id=559b3e713f5844b6ae37151d56dd419f&role.id=8051174e7a7947d5b991ebd793c2e543 => generated 608 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 10:19:22.011654 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f555199-2e0a-49b6-901b-48c7f6dd57a4 None None] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:22.012200 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f555199-2e0a-49b6-901b-48c7f6dd57a4 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:22.012200 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f555199-2e0a-49b6-901b-48c7f6dd57a4 None None] PATH_INFO: `/` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:22.013841 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 440/877] 10.222.0.49 () {54 vars in 926 bytes} [Wed Jun 15 10:19:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:19:22.019258 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d49aad-6703-406d-ae65-8cf4582a55f6 None None] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:22.019655 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d49aad-6703-406d-ae65-8cf4582a55f6 None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:22.019811 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d49aad-6703-406d-ae65-8cf4582a55f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:22.037341 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.auth.core [None req-c0d49aad-6703-406d-ae65-8cf4582a55f6 None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:19:22.038822 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-c0d49aad-6703-406d-ae65-8cf4582a55f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:19:22.107041 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 438/878] 10.222.0.49 () {58 vars in 1023 bytes} [Wed Jun 15 10:19:22 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 10:19:22.129978 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f555199-2e0a-49b6-901b-48c7f6dd57a4 None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:19:22.189673 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:19:22.241418 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f555199-2e0a-49b6-901b-48c7f6dd57a4 None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:19:22.281431 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc73b1ab-eaf8-452a-b08f-9788da8eb6d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:19:22.282255 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc73b1ab-eaf8-452a-b08f-9788da8eb6d4 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:22.282531 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc73b1ab-eaf8-452a-b08f-9788da8eb6d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:22.282803 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc73b1ab-eaf8-452a-b08f-9788da8eb6d4 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:22.283480 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc73b1ab-eaf8-452a-b08f-9788da8eb6d4 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:19:22.284835 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc73b1ab-eaf8-452a-b08f-9788da8eb6d4 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:19:22.297145 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application [None req-dc73b1ab-eaf8-452a-b08f-9788da8eb6d4 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 15 10:19:22.297145 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:19:22.297145 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 15 10:19:22.297145 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 15 10:19:22.297145 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 15 10:19:22.297145 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 15 10:19:22.297145 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 15 10:19:22.297145 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 10:19:22.297145 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 15 10:19:22.297145 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Jun 15 10:19:22.297145 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 15 10:19:22.297145 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Jun 15 10:19:22.297145 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:19:22.297145 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:19:22.297145 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:19:22.297145 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:19:22.298102 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:19:22.298102 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:19:22.298102 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:19:22.298102 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:19:22.298102 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:19:22.298102 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:19:22.298102 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:19:22.298102 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 15 10:19:22.298102 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 15 10:19:22.298102 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 15 10:19:22.298102 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 15 10:19:22.298102 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:19:22.298102 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:19:22.298102 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:19:22.298102 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:19:22.298102 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:19:22.299110 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:19:22.299110 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:19:22.299110 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application with Lock( Jun 15 10:19:22.299110 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:19:22.299110 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:19:22.299110 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:19:22.299110 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:19:22.299110 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:19:22.299110 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:19:22.299110 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:19:22.299110 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:19:22.299110 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 15 10:19:22.299110 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 10:19:22.299110 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 15 10:19:22.299110 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Jun 15 10:19:22.299110 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 441/879] 10.222.0.49 () {56 vars in 1132 bytes} [Wed Jun 15 10:19:22 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:19:22.304549 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d49aad-6703-406d-ae65-8cf4582a55f6 None None] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:19:22.345654 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=100 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:19:22.408309 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d49aad-6703-406d-ae65-8cf4582a55f6 None None] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:19:22.443349 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fff1084-e8cf-4830-b645-b425464fded9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:19:22.444291 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fff1084-e8cf-4830-b645-b425464fded9 None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:22.444767 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fff1084-e8cf-4830-b645-b425464fded9 None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:22.445223 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fff1084-e8cf-4830-b645-b425464fded9 None admin] PATH_INFO: `/v3/services` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:22.446103 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fff1084-e8cf-4830-b645-b425464fded9 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:19:22.446586 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fff1084-e8cf-4830-b645-b425464fded9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:19:22.448810 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fff1084-e8cf-4830-b645-b425464fded9 None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:19:22.454087 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 439/880] 10.222.0.49 () {56 vars in 1126 bytes} [Wed Jun 15 10:19:22 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:19:22.459832 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc73b1ab-eaf8-452a-b08f-9788da8eb6d4 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:19:22.568909 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc73b1ab-eaf8-452a-b08f-9788da8eb6d4 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:19:22.589568 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:19:22.604018 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af09764-2ba3-40f2-bc4d-ace931a0ee0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:19:22.604939 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af09764-2ba3-40f2-bc4d-ace931a0ee0b None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:22.605291 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af09764-2ba3-40f2-bc4d-ace931a0ee0b None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:22.605710 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af09764-2ba3-40f2-bc4d-ace931a0ee0b None admin] PATH_INFO: `/v3/services` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:22.606486 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af09764-2ba3-40f2-bc4d-ace931a0ee0b None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:19:22.607037 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af09764-2ba3-40f2-bc4d-ace931a0ee0b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:19:22.608436 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af09764-2ba3-40f2-bc4d-ace931a0ee0b None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:19:22.613743 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 442/881] 10.222.0.49 () {56 vars in 1126 bytes} [Wed Jun 15 10:19:22 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:19:22.620749 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fff1084-e8cf-4830-b645-b425464fded9 None admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:19:22.699988 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c58798-0c91-4d93-93d5-eba9c200abfd None None] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:22.700275 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c58798-0c91-4d93-93d5-eba9c200abfd None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:22.700475 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c58798-0c91-4d93-93d5-eba9c200abfd None None] PATH_INFO: `/` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:22.702094 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 443/882] 10.222.0.49 () {54 vars in 926 bytes} [Wed Jun 15 10:19:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:19:22.709418 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df91903-b3f3-413a-95be-1addfd46825a None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:22.709418 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df91903-b3f3-413a-95be-1addfd46825a None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:22.709418 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df91903-b3f3-413a-95be-1addfd46825a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:22.724935 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-0df91903-b3f3-413a-95be-1addfd46825a None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:19:22.726243 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-0df91903-b3f3-413a-95be-1addfd46825a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:19:22.733378 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fff1084-e8cf-4830-b645-b425464fded9 None admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:19:22.735051 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=100 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:19:22.774618 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f725483-f72f-4ec0-b195-ade31413e1d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:19:22.775726 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f725483-f72f-4ec0-b195-ade31413e1d8 None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:22.775726 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f725483-f72f-4ec0-b195-ade31413e1d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:22.776003 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f725483-f72f-4ec0-b195-ade31413e1d8 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:22.776754 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f725483-f72f-4ec0-b195-ade31413e1d8 None admin] RBAC: Adding query filter params (interface=public, service_id=3fe71d501d7449f6a8711c2e0ac41b9d, region_id=RegionOne) {{(pid=80119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:19:22.777063 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f725483-f72f-4ec0-b195-ade31413e1d8 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:19:22.778437 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f725483-f72f-4ec0-b195-ade31413e1d8 None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:19:22.787028 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 440/883] 10.222.0.49 () {56 vars in 1251 bytes} [Wed Jun 15 10:19:22 2022] GET /identity/v3/endpoints?service_id=3fe71d501d7449f6a8711c2e0ac41b9d&interface=public®ion_id=RegionOne => generated 194 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:19:22.795021 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f725483-f72f-4ec0-b195-ade31413e1d8 None admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:19:22.819111 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 444/884] 10.222.0.49 () {58 vars in 1023 bytes} [Wed Jun 15 10:19:22 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 15 10:19:22.837905 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df91903-b3f3-413a-95be-1addfd46825a None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:19:22.920378 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f725483-f72f-4ec0-b195-ade31413e1d8 None admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:19:22.963894 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df91903-b3f3-413a-95be-1addfd46825a None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:19:22.972195 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430da0ac-39d7-4307-9f1f-760e0de78ed7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:19:22.973162 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430da0ac-39d7-4307-9f1f-760e0de78ed7 None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:22.974000 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430da0ac-39d7-4307-9f1f-760e0de78ed7 None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:22.974493 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430da0ac-39d7-4307-9f1f-760e0de78ed7 None admin] PATH_INFO: `/v3/services` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:22.976095 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-430da0ac-39d7-4307-9f1f-760e0de78ed7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:19:22.977895 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-430da0ac-39d7-4307-9f1f-760e0de78ed7 None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:19:22.978851 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:19:22.986993 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 441/885] 10.222.0.49 () {56 vars in 1087 bytes} [Wed Jun 15 10:19:22 2022] GET /identity/v3/services => generated 1268 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 10:19:23.006913 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78cb1cba-3974-4ebc-af54-7d878befb190 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:19:23.007792 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78cb1cba-3974-4ebc-af54-7d878befb190 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:23.008078 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78cb1cba-3974-4ebc-af54-7d878befb190 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:23.008517 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78cb1cba-3974-4ebc-af54-7d878befb190 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:23.009418 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78cb1cba-3974-4ebc-af54-7d878befb190 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:19:23.011009 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78cb1cba-3974-4ebc-af54-7d878befb190 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:19:23.024906 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application [None req-78cb1cba-3974-4ebc-af54-7d878befb190 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 15 10:19:23.024906 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:19:23.024906 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 15 10:19:23.024906 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 15 10:19:23.024906 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 15 10:19:23.024906 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 15 10:19:23.024906 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 15 10:19:23.024906 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 10:19:23.024906 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 15 10:19:23.024906 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Jun 15 10:19:23.024906 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 15 10:19:23.024906 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Jun 15 10:19:23.024906 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:19:23.024906 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:19:23.024906 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:19:23.024906 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:19:23.024906 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:19:23.027020 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:19:23.027020 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:19:23.027020 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:19:23.027020 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:19:23.027020 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:19:23.027020 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:19:23.027020 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 15 10:19:23.027020 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 15 10:19:23.027020 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 15 10:19:23.027020 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 15 10:19:23.027020 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:19:23.027020 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:19:23.027020 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:19:23.027020 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:19:23.027020 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:19:23.027020 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:19:23.028008 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:19:23.028008 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application with Lock( Jun 15 10:19:23.028008 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:19:23.028008 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:19:23.028008 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:19:23.028008 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:19:23.028008 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:19:23.028008 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:19:23.028008 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:19:23.028008 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:19:23.028008 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 15 10:19:23.028008 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 10:19:23.028008 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 15 10:19:23.028008 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Jun 15 10:19:23.028008 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 445/886] 10.222.0.49 () {56 vars in 1114 bytes} [Wed Jun 15 10:19:22 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:19:23.035550 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430da0ac-39d7-4307-9f1f-760e0de78ed7 None admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:19:23.073092 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=100 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:19:23.155615 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430da0ac-39d7-4307-9f1f-760e0de78ed7 None admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:19:23.192213 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5300a69-c236-4e7d-bff3-d296b8b22cb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:19:23.193138 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5300a69-c236-4e7d-bff3-d296b8b22cb5 None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:23.193596 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5300a69-c236-4e7d-bff3-d296b8b22cb5 None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:23.194028 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5300a69-c236-4e7d-bff3-d296b8b22cb5 None admin] PATH_INFO: `/v3/services` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:23.194886 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5300a69-c236-4e7d-bff3-d296b8b22cb5 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:19:23.195407 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5300a69-c236-4e7d-bff3-d296b8b22cb5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:19:23.196965 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5300a69-c236-4e7d-bff3-d296b8b22cb5 None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:19:23.202421 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 442/887] 10.222.0.49 () {56 vars in 1114 bytes} [Wed Jun 15 10:19:23 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:19:23.208228 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78cb1cba-3974-4ebc-af54-7d878befb190 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:19:23.209973 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb998e7-4802-4b70-bbf2-a1306955007d None None] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:23.210265 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb998e7-4802-4b70-bbf2-a1306955007d None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:23.210544 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb998e7-4802-4b70-bbf2-a1306955007d None None] PATH_INFO: `/` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:23.212118 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 443/888] 10.222.0.49 () {54 vars in 926 bytes} [Wed Jun 15 10:19:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:19:23.220069 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46381c3-3f2e-42fd-bc97-587e4ada93a3 None None] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:23.220304 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46381c3-3f2e-42fd-bc97-587e4ada93a3 None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:23.220573 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46381c3-3f2e-42fd-bc97-587e4ada93a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:23.242313 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.auth.core [None req-c46381c3-3f2e-42fd-bc97-587e4ada93a3 None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:19:23.244270 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-c46381c3-3f2e-42fd-bc97-587e4ada93a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:19:23.335491 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78cb1cba-3974-4ebc-af54-7d878befb190 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:19:23.343128 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 444/889] 10.222.0.49 () {58 vars in 1023 bytes} [Wed Jun 15 10:19:23 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 15 10:19:23.345573 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2658e0-9e15-48e4-83ce-25be7588cf71 None None] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:23.345866 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2658e0-9e15-48e4-83ce-25be7588cf71 None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:23.346147 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2658e0-9e15-48e4-83ce-25be7588cf71 None None] PATH_INFO: `/` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:23.347602 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 445/890] 10.222.0.49 () {54 vars in 926 bytes} [Wed Jun 15 10:19:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:19:23.356226 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267ed333-afc4-4f7f-9f42-48162628f443 None None] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:23.356518 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267ed333-afc4-4f7f-9f42-48162628f443 None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:23.356797 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267ed333-afc4-4f7f-9f42-48162628f443 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:23.379296 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.auth.core [None req-267ed333-afc4-4f7f-9f42-48162628f443 None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:19:23.381021 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-267ed333-afc4-4f7f-9f42-48162628f443 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:19:23.381582 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216c1063-c6e8-49da-96df-32e619dfeb62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:19:23.382253 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216c1063-c6e8-49da-96df-32e619dfeb62 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:23.382535 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216c1063-c6e8-49da-96df-32e619dfeb62 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:23.382808 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216c1063-c6e8-49da-96df-32e619dfeb62 None admin] PATH_INFO: `/v3/services` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:23.383541 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216c1063-c6e8-49da-96df-32e619dfeb62 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:19:23.383845 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216c1063-c6e8-49da-96df-32e619dfeb62 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:19:23.385748 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216c1063-c6e8-49da-96df-32e619dfeb62 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:19:23.390893 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 446/891] 10.222.0.49 () {56 vars in 1114 bytes} [Wed Jun 15 10:19:23 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:19:23.391778 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216c1063-c6e8-49da-96df-32e619dfeb62 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:19:23.411694 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=100 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:19:23.438460 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:19:23.465818 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 446/892] 10.222.0.49 () {58 vars in 1023 bytes} [Wed Jun 15 10:19:23 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 15 10:19:23.466560 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267ed333-afc4-4f7f-9f42-48162628f443 None None] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:19:23.506203 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216c1063-c6e8-49da-96df-32e619dfeb62 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:19:23.543203 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e404226-9ef5-4c8a-b8b4-c9b59a1c7a25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:19:23.544170 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e404226-9ef5-4c8a-b8b4-c9b59a1c7a25 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:23.544459 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e404226-9ef5-4c8a-b8b4-c9b59a1c7a25 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:23.544729 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e404226-9ef5-4c8a-b8b4-c9b59a1c7a25 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:23.555636 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e404226-9ef5-4c8a-b8b4-c9b59a1c7a25 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 10:19:23.563881 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application [None req-0e404226-9ef5-4c8a-b8b4-c9b59a1c7a25 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 15 10:19:23.563881 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:19:23.563881 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:19:23.563881 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:19:23.563881 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:19:23.563881 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:19:23.563881 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:19:23.563881 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:19:23.563881 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:19:23.563881 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:19:23.563881 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:19:23.563881 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:19:23.563881 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 10:19:23.563881 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 10:19:23.563881 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 10:19:23.563881 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 10:19:23.564860 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 10:19:23.564860 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 10:19:23.564860 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 10:19:23.564860 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 10:19:23.564860 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:19:23.564860 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:19:23.564860 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:19:23.564860 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:19:23.564860 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:19:23.564860 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:19:23.564860 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:19:23.564860 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application with Lock( Jun 15 10:19:23.564860 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:19:23.564860 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:19:23.564860 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:19:23.564860 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:19:23.565718 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:19:23.565718 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:19:23.565718 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:19:23.565718 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:19:23.565718 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 10:19:23.565718 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 10:19:23.565718 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 10:19:23.565718 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 10:19:23.565718 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 10:19:23.565718 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 10:19:23.565718 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 15 10:19:23.565718 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Jun 15 10:19:23.565718 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 447/893] 10.222.0.49 () {56 vars in 1120 bytes} [Wed Jun 15 10:19:23 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:19:23.566438 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e404226-9ef5-4c8a-b8b4-c9b59a1c7a25 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:19:23.591893 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267ed333-afc4-4f7f-9f42-48162628f443 None None] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:19:23.629082 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f1de16-2b12-49cc-99e9-e86f25201da8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:19:23.629751 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f1de16-2b12-49cc-99e9-e86f25201da8 None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:23.630062 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f1de16-2b12-49cc-99e9-e86f25201da8 None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:23.630373 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f1de16-2b12-49cc-99e9-e86f25201da8 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:23.631120 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f1de16-2b12-49cc-99e9-e86f25201da8 None admin] RBAC: Adding query filter params (interface=public, service_id=68f6d7c0e32e4aea9a96b0e86bee676b, region_id=RegionOne) {{(pid=80119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:19:23.631449 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f1de16-2b12-49cc-99e9-e86f25201da8 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:19:23.632810 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f1de16-2b12-49cc-99e9-e86f25201da8 None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:19:23.643177 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 447/894] 10.222.0.49 () {56 vars in 1251 bytes} [Wed Jun 15 10:19:23 2022] GET /identity/v3/endpoints?service_id=68f6d7c0e32e4aea9a96b0e86bee676b&interface=public®ion_id=RegionOne => generated 194 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:19:23.644253 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f1de16-2b12-49cc-99e9-e86f25201da8 None admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:19:23.665090 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=100 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:19:23.677692 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e404226-9ef5-4c8a-b8b4-c9b59a1c7a25 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:19:23.679809 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:19:23.716339 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1475a93-9599-4353-b815-8a724a261e65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:19:23.717155 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1475a93-9599-4353-b815-8a724a261e65 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:23.717445 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1475a93-9599-4353-b815-8a724a261e65 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:23.717725 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1475a93-9599-4353-b815-8a724a261e65 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:23.718380 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1475a93-9599-4353-b815-8a724a261e65 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:19:23.719805 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1475a93-9599-4353-b815-8a724a261e65 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:19:23.734074 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application [None req-a1475a93-9599-4353-b815-8a724a261e65 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 15 10:19:23.734074 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:19:23.734074 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 15 10:19:23.734074 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 15 10:19:23.734074 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 15 10:19:23.734074 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 15 10:19:23.734074 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 15 10:19:23.734074 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 10:19:23.734074 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 15 10:19:23.734074 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Jun 15 10:19:23.734074 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 15 10:19:23.734074 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Jun 15 10:19:23.734074 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:19:23.734074 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:19:23.734074 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:19:23.734074 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:19:23.734074 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:19:23.735145 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:19:23.735145 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:19:23.735145 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:19:23.735145 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:19:23.735145 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:19:23.735145 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:19:23.735145 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 15 10:19:23.735145 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 15 10:19:23.735145 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 15 10:19:23.735145 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 15 10:19:23.735145 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:19:23.735145 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:19:23.735145 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:19:23.735145 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:19:23.735145 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:19:23.735145 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:19:23.735981 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:19:23.735981 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application with Lock( Jun 15 10:19:23.735981 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:19:23.735981 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:19:23.735981 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:19:23.735981 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:19:23.735981 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:19:23.735981 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:19:23.735981 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:19:23.735981 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:19:23.735981 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 15 10:19:23.735981 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 10:19:23.735981 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 15 10:19:23.735981 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Jun 15 10:19:23.735981 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 448/895] 10.222.0.49 () {56 vars in 1126 bytes} [Wed Jun 15 10:19:23 2022] GET /identity/v3/services/object-store => generated 93 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:19:23.736815 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1475a93-9599-4353-b815-8a724a261e65 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:19:23.755477 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f1de16-2b12-49cc-99e9-e86f25201da8 None admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:19:23.800193 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c09d96-7165-48a4-a339-b86f8b6545d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:19:23.800946 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c09d96-7165-48a4-a339-b86f8b6545d6 None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:23.801246 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c09d96-7165-48a4-a339-b86f8b6545d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:23.801494 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c09d96-7165-48a4-a339-b86f8b6545d6 None admin] PATH_INFO: `/v3/roles` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:23.802243 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c09d96-7165-48a4-a339-b86f8b6545d6 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:19:23.802537 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c09d96-7165-48a4-a339-b86f8b6545d6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:19:23.804055 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c09d96-7165-48a4-a339-b86f8b6545d6 None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:19:23.813677 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 448/896] 10.222.0.49 () {56 vars in 1115 bytes} [Wed Jun 15 10:19:23 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:19:23.818840 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c09d96-7165-48a4-a339-b86f8b6545d6 None admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:19:23.842733 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1475a93-9599-4353-b815-8a724a261e65 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:19:23.882857 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187179bd-b217-4334-9722-d1fb0ea8dc14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:19:23.883559 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187179bd-b217-4334-9722-d1fb0ea8dc14 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:23.883847 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187179bd-b217-4334-9722-d1fb0ea8dc14 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:23.884125 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187179bd-b217-4334-9722-d1fb0ea8dc14 None admin] PATH_INFO: `/v3/services` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:23.885403 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-187179bd-b217-4334-9722-d1fb0ea8dc14 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:19:23.886715 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-187179bd-b217-4334-9722-d1fb0ea8dc14 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:19:23.892467 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 449/897] 10.222.0.49 () {56 vars in 1087 bytes} [Wed Jun 15 10:19:23 2022] GET /identity/v3/services => generated 1268 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 10:19:23.893482 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187179bd-b217-4334-9722-d1fb0ea8dc14 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:19:23.911482 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=100 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:19:23.934777 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:19:23.936339 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c09d96-7165-48a4-a339-b86f8b6545d6 None admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:19:23.982341 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3501b6-466f-42b1-b813-3fc1c163acf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:19:23.983278 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3501b6-466f-42b1-b813-3fc1c163acf7 None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:23.983658 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3501b6-466f-42b1-b813-3fc1c163acf7 None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:23.984044 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3501b6-466f-42b1-b813-3fc1c163acf7 None admin] PATH_INFO: `/v3/services` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:23.984882 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a3501b6-466f-42b1-b813-3fc1c163acf7 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:19:23.985996 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a3501b6-466f-42b1-b813-3fc1c163acf7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:19:23.986836 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a3501b6-466f-42b1-b813-3fc1c163acf7 None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:19:23.992931 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 449/898] 10.222.0.49 () {56 vars in 1122 bytes} [Wed Jun 15 10:19:23 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:19:23.998945 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3501b6-466f-42b1-b813-3fc1c163acf7 None admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:19:24.020000 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187179bd-b217-4334-9722-d1fb0ea8dc14 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:19:24.058942 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a1d192-a1d5-4394-a345-e1022205d281 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:19:24.060639 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a1d192-a1d5-4394-a345-e1022205d281 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:24.061148 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a1d192-a1d5-4394-a345-e1022205d281 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:24.061522 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a1d192-a1d5-4394-a345-e1022205d281 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:24.074274 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a1d192-a1d5-4394-a345-e1022205d281 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:19:24.075938 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a1d192-a1d5-4394-a345-e1022205d281 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:19:24.095010 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application [None req-52a1d192-a1d5-4394-a345-e1022205d281 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 10:19:24.095010 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:19:24.095010 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:19:24.095010 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:19:24.095010 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:19:24.095010 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:19:24.095010 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:19:24.095010 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:19:24.095010 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:19:24.095010 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:19:24.095010 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:19:24.095010 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:19:24.095010 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 10:19:24.095010 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 10:19:24.095010 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 10:19:24.096137 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 10:19:24.096137 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:19:24.096137 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:19:24.096137 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:19:24.096137 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:19:24.096137 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:19:24.096137 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:19:24.096137 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:19:24.096137 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application with Lock( Jun 15 10:19:24.096137 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:19:24.096137 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:19:24.096137 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:19:24.096137 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:19:24.096137 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:19:24.096137 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:19:24.096137 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:19:24.096137 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:19:24.097080 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 10:19:24.097080 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 10:19:24.097080 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 10:19:24.097080 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Jun 15 10:19:24.097623 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 450/899] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Jun 15 10:19:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:19:24.103857 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a1d192-a1d5-4394-a345-e1022205d281 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:19:24.138282 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3501b6-466f-42b1-b813-3fc1c163acf7 None admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:19:24.178112 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90227bf8-5f96-43f3-90ee-26bbc688985b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:19:24.179455 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90227bf8-5f96-43f3-90ee-26bbc688985b None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:24.180092 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90227bf8-5f96-43f3-90ee-26bbc688985b None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:24.180960 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90227bf8-5f96-43f3-90ee-26bbc688985b None admin] PATH_INFO: `/v3/services` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:24.182678 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90227bf8-5f96-43f3-90ee-26bbc688985b None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:19:24.183368 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90227bf8-5f96-43f3-90ee-26bbc688985b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:19:24.185651 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90227bf8-5f96-43f3-90ee-26bbc688985b None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:19:24.198423 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 450/900] 10.222.0.49 () {56 vars in 1122 bytes} [Wed Jun 15 10:19:23 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:19:24.224757 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a1d192-a1d5-4394-a345-e1022205d281 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:19:24.248210 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:19:24.268764 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43a6c38-c8a8-495a-9f31-fc618edae02e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:19:24.269463 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43a6c38-c8a8-495a-9f31-fc618edae02e None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:24.269753 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43a6c38-c8a8-495a-9f31-fc618edae02e None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:24.270032 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43a6c38-c8a8-495a-9f31-fc618edae02e None admin] PATH_INFO: `/v3/domains` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:24.270778 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d43a6c38-c8a8-495a-9f31-fc618edae02e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:19:24.271106 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d43a6c38-c8a8-495a-9f31-fc618edae02e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:19:24.272518 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d43a6c38-c8a8-495a-9f31-fc618edae02e None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:19:24.286310 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 451/901] 10.222.0.49 () {56 vars in 1109 bytes} [Wed Jun 15 10:19:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:19:24.292192 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90227bf8-5f96-43f3-90ee-26bbc688985b None admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:19:24.304892 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=100 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:19:24.407642 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90227bf8-5f96-43f3-90ee-26bbc688985b None admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:19:24.444265 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08112510-60b9-4023-902b-5994cbf40ccf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:19:24.445140 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08112510-60b9-4023-902b-5994cbf40ccf None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:24.445431 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08112510-60b9-4023-902b-5994cbf40ccf None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:24.445708 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08112510-60b9-4023-902b-5994cbf40ccf None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:24.454149 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08112510-60b9-4023-902b-5994cbf40ccf None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:19:24.455631 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08112510-60b9-4023-902b-5994cbf40ccf None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:19:24.470802 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application [None req-08112510-60b9-4023-902b-5994cbf40ccf None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 15 10:19:24.470802 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:19:24.470802 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:19:24.470802 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:19:24.470802 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:19:24.470802 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:19:24.470802 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:19:24.470802 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:19:24.470802 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:19:24.470802 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:19:24.470802 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:19:24.470802 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:19:24.470802 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 15 10:19:24.470802 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 15 10:19:24.470802 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 15 10:19:24.470802 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 15 10:19:24.471810 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:19:24.471810 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:19:24.471810 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 15 10:19:24.471810 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 10:19:24.471810 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 15 10:19:24.471810 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 10:19:24.471810 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:19:24.471810 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:19:24.471810 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:19:24.471810 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:19:24.471810 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:19:24.471810 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application with Lock( Jun 15 10:19:24.471810 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:19:24.471810 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:19:24.471810 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:19:24.471810 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:19:24.471810 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:19:24.471810 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:19:24.473359 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:19:24.473359 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:19:24.473359 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 15 10:19:24.473359 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 15 10:19:24.473359 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 15 10:19:24.473359 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 15 10:19:24.473359 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 15 10:19:24.473359 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 15 10:19:24.473359 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 15 10:19:24.473359 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application Jun 15 10:19:24.473359 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 451/902] 10.222.0.49 () {56 vars in 1117 bytes} [Wed Jun 15 10:19:24 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:19:24.477922 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43a6c38-c8a8-495a-9f31-fc618edae02e None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:19:24.580260 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43a6c38-c8a8-495a-9f31-fc618edae02e None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:19:24.614738 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb10829-d2fb-4a2d-a31a-84afce273518 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:19:24.615451 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb10829-d2fb-4a2d-a31a-84afce273518 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:24.615734 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb10829-d2fb-4a2d-a31a-84afce273518 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:24.616011 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb10829-d2fb-4a2d-a31a-84afce273518 None admin] PATH_INFO: `/v3/users` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:24.616853 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb10829-d2fb-4a2d-a31a-84afce273518 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:19:24.617162 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb10829-d2fb-4a2d-a31a-84afce273518 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:19:24.618580 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb10829-d2fb-4a2d-a31a-84afce273518 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:19:24.635544 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:19:24.637723 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 452/903] 10.222.0.49 () {56 vars in 1149 bytes} [Wed Jun 15 10:19:24 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:19:24.643886 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08112510-60b9-4023-902b-5994cbf40ccf None admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:19:24.722220 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=100 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:19:24.760343 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08112510-60b9-4023-902b-5994cbf40ccf None admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:19:24.802898 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82dc65a-06a9-4582-a0c3-2ab4ce872516 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:19:24.803613 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82dc65a-06a9-4582-a0c3-2ab4ce872516 None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:24.803899 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82dc65a-06a9-4582-a0c3-2ab4ce872516 None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:24.804179 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82dc65a-06a9-4582-a0c3-2ab4ce872516 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:24.872876 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e82dc65a-06a9-4582-a0c3-2ab4ce872516 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:19:24.874338 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e82dc65a-06a9-4582-a0c3-2ab4ce872516 None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:19:24.939548 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=100 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:19:24.975855 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 452/904] 10.222.0.49 () {58 vars in 1282 bytes} [Wed Jun 15 10:19:24 2022] GET /identity/v3/auth/tokens => generated 1362 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 15 10:19:24.981626 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb10829-d2fb-4a2d-a31a-84afce273518 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:19:25.084114 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb10829-d2fb-4a2d-a31a-84afce273518 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:19:25.122321 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05776673-e466-443a-ae1b-67cce7844d8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:19:25.123153 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05776673-e466-443a-ae1b-67cce7844d8c None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:25.123443 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05776673-e466-443a-ae1b-67cce7844d8c None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:25.123720 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05776673-e466-443a-ae1b-67cce7844d8c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:25.134215 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05776673-e466-443a-ae1b-67cce7844d8c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:19:25.135745 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05776673-e466-443a-ae1b-67cce7844d8c None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:19:25.153337 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application [None req-05776673-e466-443a-ae1b-67cce7844d8c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 10:19:25.153337 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:19:25.153337 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:19:25.153337 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:19:25.153337 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:19:25.153337 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:19:25.153337 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:19:25.153337 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:19:25.153337 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:19:25.153337 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:19:25.153337 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:19:25.153337 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:19:25.153337 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 10:19:25.153337 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 10:19:25.153337 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 10:19:25.154239 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 10:19:25.154239 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:19:25.154239 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:19:25.154239 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:19:25.154239 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:19:25.154239 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:19:25.154239 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:19:25.154239 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:19:25.154239 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application with Lock( Jun 15 10:19:25.154239 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:19:25.154239 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:19:25.154239 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:19:25.154239 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:19:25.154239 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:19:25.154239 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:19:25.154239 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:19:25.154239 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:19:25.155185 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 10:19:25.155185 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 10:19:25.155185 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 10:19:25.155185 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Jun 15 10:19:25.155185 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 453/905] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Jun 15 10:19:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:19:25.160409 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82dc65a-06a9-4582-a0c3-2ab4ce872516 None admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:19:25.269160 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82dc65a-06a9-4582-a0c3-2ab4ce872516 None admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:19:25.275712 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40c7d1e-ee1a-4a1d-a9e9-67fd63990242 None None] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:25.276019 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40c7d1e-ee1a-4a1d-a9e9-67fd63990242 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:25.276296 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40c7d1e-ee1a-4a1d-a9e9-67fd63990242 None None] PATH_INFO: `/` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:25.277934 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 454/906] 10.222.0.49 () {54 vars in 926 bytes} [Wed Jun 15 10:19:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:19:25.282821 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ea0b6f-824d-4f3b-b8e4-3a4ad9ccd9eb None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:25.283129 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ea0b6f-824d-4f3b-b8e4-3a4ad9ccd9eb None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:25.283419 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ea0b6f-824d-4f3b-b8e4-3a4ad9ccd9eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:25.300948 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-06ea0b6f-824d-4f3b-b8e4-3a4ad9ccd9eb None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:19:25.302148 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-06ea0b6f-824d-4f3b-b8e4-3a4ad9ccd9eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:19:25.304586 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b059de3b-8d82-43a6-a85e-ba84d2d1d38d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:19:25.305371 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b059de3b-8d82-43a6-a85e-ba84d2d1d38d None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:25.305656 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b059de3b-8d82-43a6-a85e-ba84d2d1d38d None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:25.305934 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b059de3b-8d82-43a6-a85e-ba84d2d1d38d None admin] PATH_INFO: `/v3/domains` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:25.307203 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b059de3b-8d82-43a6-a85e-ba84d2d1d38d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:19:25.307515 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b059de3b-8d82-43a6-a85e-ba84d2d1d38d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:19:25.308893 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b059de3b-8d82-43a6-a85e-ba84d2d1d38d None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:19:25.326619 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 453/907] 10.222.0.49 () {56 vars in 1109 bytes} [Wed Jun 15 10:19:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:19:25.327637 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b059de3b-8d82-43a6-a85e-ba84d2d1d38d None admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:19:25.356086 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=100 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:19:25.357234 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:19:25.391109 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 455/908] 10.222.0.49 () {58 vars in 1023 bytes} [Wed Jun 15 10:19:25 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 10:19:25.408024 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ea0b6f-824d-4f3b-b8e4-3a4ad9ccd9eb None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:19:25.441360 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b059de3b-8d82-43a6-a85e-ba84d2d1d38d None admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:19:25.480912 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ce84d9-a61f-46d2-b6a9-d6cfcb2f7d0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:19:25.481713 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ce84d9-a61f-46d2-b6a9-d6cfcb2f7d0e None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:25.482002 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ce84d9-a61f-46d2-b6a9-d6cfcb2f7d0e None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:25.482282 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ce84d9-a61f-46d2-b6a9-d6cfcb2f7d0e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:25.489248 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ce84d9-a61f-46d2-b6a9-d6cfcb2f7d0e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:19:25.490705 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ce84d9-a61f-46d2-b6a9-d6cfcb2f7d0e None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:19:25.506002 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application [None req-73ce84d9-a61f-46d2-b6a9-d6cfcb2f7d0e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 10:19:25.506002 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:19:25.506002 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:19:25.506002 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:19:25.506002 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:19:25.506002 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:19:25.506002 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:19:25.506002 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:19:25.506002 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:19:25.506002 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:19:25.506002 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:19:25.506002 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:19:25.506002 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 15 10:19:25.506002 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 15 10:19:25.506002 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 15 10:19:25.506974 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 15 10:19:25.506974 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:19:25.506974 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:19:25.506974 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:19:25.506974 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:19:25.506974 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:19:25.506974 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:19:25.506974 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:19:25.506974 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application with Lock( Jun 15 10:19:25.506974 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:19:25.506974 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:19:25.506974 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:19:25.506974 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:19:25.506974 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:19:25.506974 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:19:25.506974 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:19:25.506974 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:19:25.508215 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 15 10:19:25.508215 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 15 10:19:25.508215 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 15 10:19:25.508215 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 15 10:19:25.508215 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 15 10:19:25.508215 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 15 10:19:25.508215 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 10:19:25.508215 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application Jun 15 10:19:25.512724 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 454/909] 10.222.0.49 () {56 vars in 1111 bytes} [Wed Jun 15 10:19:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:19:25.513703 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ce84d9-a61f-46d2-b6a9-d6cfcb2f7d0e None admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:19:25.533946 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ea0b6f-824d-4f3b-b8e4-3a4ad9ccd9eb None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:19:25.577847 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3716fa2-3366-4f85-87f5-260b900bd8b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:19:25.579331 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3716fa2-3366-4f85-87f5-260b900bd8b5 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:25.580085 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3716fa2-3366-4f85-87f5-260b900bd8b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:25.580961 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3716fa2-3366-4f85-87f5-260b900bd8b5 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:25.582104 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3716fa2-3366-4f85-87f5-260b900bd8b5 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:19:25.584003 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3716fa2-3366-4f85-87f5-260b900bd8b5 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:19:25.601283 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application [None req-a3716fa2-3366-4f85-87f5-260b900bd8b5 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 15 10:19:25.601283 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:19:25.601283 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 15 10:19:25.601283 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 15 10:19:25.601283 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 15 10:19:25.601283 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 15 10:19:25.601283 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 15 10:19:25.601283 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 10:19:25.601283 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 15 10:19:25.601283 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Jun 15 10:19:25.601283 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 15 10:19:25.601283 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Jun 15 10:19:25.601283 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:19:25.601283 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:19:25.601283 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:19:25.601283 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:19:25.602334 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:19:25.602334 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:19:25.602334 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:19:25.602334 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:19:25.602334 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:19:25.602334 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:19:25.602334 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:19:25.602334 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 15 10:19:25.602334 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 15 10:19:25.602334 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 15 10:19:25.602334 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 15 10:19:25.602334 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:19:25.602334 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:19:25.602334 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:19:25.602334 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:19:25.602334 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:19:25.603269 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:19:25.603269 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:19:25.603269 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application with Lock( Jun 15 10:19:25.603269 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:19:25.603269 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:19:25.603269 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:19:25.603269 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:19:25.603269 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:19:25.603269 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:19:25.603269 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:19:25.603269 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:19:25.603269 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 15 10:19:25.603269 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 10:19:25.603269 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 15 10:19:25.603269 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Jun 15 10:19:25.609593 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 456/910] 10.222.0.49 () {56 vars in 1132 bytes} [Wed Jun 15 10:19:25 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:19:25.611635 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3716fa2-3366-4f85-87f5-260b900bd8b5 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:19:25.616536 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=100 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:19:25.636187 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ce84d9-a61f-46d2-b6a9-d6cfcb2f7d0e None admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:19:25.665389 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:19:25.683171 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30a1445-4cfa-4ffb-b65a-9bc1aa6cc259 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:19:25.684705 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30a1445-4cfa-4ffb-b65a-9bc1aa6cc259 None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:25.685226 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30a1445-4cfa-4ffb-b65a-9bc1aa6cc259 None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:25.685710 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30a1445-4cfa-4ffb-b65a-9bc1aa6cc259 None admin] PATH_INFO: `/v3/projects` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:25.686723 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30a1445-4cfa-4ffb-b65a-9bc1aa6cc259 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:19:25.687413 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30a1445-4cfa-4ffb-b65a-9bc1aa6cc259 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:19:25.689847 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30a1445-4cfa-4ffb-b65a-9bc1aa6cc259 None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:19:25.705534 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 455/911] 10.222.0.49 () {56 vars in 1148 bytes} [Wed Jun 15 10:19:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:19:25.711512 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30a1445-4cfa-4ffb-b65a-9bc1aa6cc259 None admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:19:25.733254 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3716fa2-3366-4f85-87f5-260b900bd8b5 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:19:25.784278 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b44199-ba58-4faa-8ce2-cc5e3d6b0a15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:19:25.785423 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b44199-ba58-4faa-8ce2-cc5e3d6b0a15 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:25.785860 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b44199-ba58-4faa-8ce2-cc5e3d6b0a15 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:25.786298 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b44199-ba58-4faa-8ce2-cc5e3d6b0a15 None admin] PATH_INFO: `/v3/services` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:25.787436 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b44199-ba58-4faa-8ce2-cc5e3d6b0a15 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:19:25.787916 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b44199-ba58-4faa-8ce2-cc5e3d6b0a15 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:19:25.792697 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b44199-ba58-4faa-8ce2-cc5e3d6b0a15 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:19:25.805496 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 457/912] 10.222.0.49 () {56 vars in 1126 bytes} [Wed Jun 15 10:19:25 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:19:25.806375 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b44199-ba58-4faa-8ce2-cc5e3d6b0a15 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:19:25.834459 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30a1445-4cfa-4ffb-b65a-9bc1aa6cc259 None admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:19:25.870672 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152ad6af-7e7a-4e05-810b-5e22edd27b2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:19:25.871515 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152ad6af-7e7a-4e05-810b-5e22edd27b2d None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:25.871892 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152ad6af-7e7a-4e05-810b-5e22edd27b2d None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:25.872275 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152ad6af-7e7a-4e05-810b-5e22edd27b2d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:25.873189 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-152ad6af-7e7a-4e05-810b-5e22edd27b2d None admin] RBAC: Adding query filter params (role.id=d962cdb5fdb24466a560f5d7bcb1e034, scope.project.id=559b3e713f5844b6ae37151d56dd419f, user.id=103e10244eca444b86b9c51ff314d445) {{(pid=80119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:19:25.873664 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-152ad6af-7e7a-4e05-810b-5e22edd27b2d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:19:25.875262 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-152ad6af-7e7a-4e05-810b-5e22edd27b2d None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:19:25.887254 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 456/913] 10.222.0.49 () {56 vars in 1374 bytes} [Wed Jun 15 10:19:25 2022] GET /identity/v3/role_assignments?user.id=103e10244eca444b86b9c51ff314d445&scope.project.id=559b3e713f5844b6ae37151d56dd419f&role.id=d962cdb5fdb24466a560f5d7bcb1e034 => generated 259 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 10:19:25.935790 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b44199-ba58-4faa-8ce2-cc5e3d6b0a15 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:19:25.949042 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:19:25.976526 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a98f27-33a1-49eb-9e8b-276541018393 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:19:25.977482 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a98f27-33a1-49eb-9e8b-276541018393 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:25.977922 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a98f27-33a1-49eb-9e8b-276541018393 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:25.978421 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a98f27-33a1-49eb-9e8b-276541018393 None admin] PATH_INFO: `/v3/services` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:25.979340 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a98f27-33a1-49eb-9e8b-276541018393 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:19:25.979822 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a98f27-33a1-49eb-9e8b-276541018393 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:19:25.981371 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a98f27-33a1-49eb-9e8b-276541018393 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:19:25.987666 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 458/914] 10.222.0.49 () {56 vars in 1126 bytes} [Wed Jun 15 10:19:25 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:19:25.993580 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152ad6af-7e7a-4e05-810b-5e22edd27b2d None admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:19:26.011872 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=100 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:19:26.110044 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152ad6af-7e7a-4e05-810b-5e22edd27b2d None admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:19:26.149723 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d275c6bc-dfb8-41fc-a359-2630f827716f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:19:26.150423 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d275c6bc-dfb8-41fc-a359-2630f827716f None admin] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:26.150706 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d275c6bc-dfb8-41fc-a359-2630f827716f None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:26.150984 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d275c6bc-dfb8-41fc-a359-2630f827716f None admin] PATH_INFO: `/v3/endpoints` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:26.151754 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d275c6bc-dfb8-41fc-a359-2630f827716f None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:19:26.153188 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d275c6bc-dfb8-41fc-a359-2630f827716f None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:19:26.170084 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14c95f9-fe38-4b2e-92fb-77fa81594218 None None] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:26.170619 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14c95f9-fe38-4b2e-92fb-77fa81594218 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:26.170619 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14c95f9-fe38-4b2e-92fb-77fa81594218 None None] PATH_INFO: `/` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:26.172363 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 459/915] 10.222.0.49 () {54 vars in 926 bytes} [Wed Jun 15 10:19:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 15 10:19:26.176364 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 457/916] 10.222.0.49 () {60 vars in 1144 bytes} [Wed Jun 15 10:19:25 2022] POST /identity/v3/endpoints => generated 347 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 10:19:26.177819 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335567cd-df37-48f3-9ba4-fb57e236e48d None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:26.178110 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335567cd-df37-48f3-9ba4-fb57e236e48d None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:26.178391 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335567cd-df37-48f3-9ba4-fb57e236e48d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:26.197759 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-335567cd-df37-48f3-9ba4-fb57e236e48d None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:19:26.199009 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-335567cd-df37-48f3-9ba4-fb57e236e48d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:19:26.279049 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 460/917] 10.222.0.49 () {58 vars in 1023 bytes} [Wed Jun 15 10:19:26 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (3 switches on core 0) Jun 15 10:19:26.303138 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d275c6bc-dfb8-41fc-a359-2630f827716f None admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:19:26.378147 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:19:26.416463 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d275c6bc-dfb8-41fc-a359-2630f827716f None admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:19:26.456663 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a8aced-fc1e-4323-9eeb-9cb251aa6d6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:19:26.457895 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a8aced-fc1e-4323-9eeb-9cb251aa6d6e None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:26.458383 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a8aced-fc1e-4323-9eeb-9cb251aa6d6e None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:26.458871 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a8aced-fc1e-4323-9eeb-9cb251aa6d6e None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:26.459795 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a8aced-fc1e-4323-9eeb-9cb251aa6d6e None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:19:26.461423 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a8aced-fc1e-4323-9eeb-9cb251aa6d6e None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:19:26.475233 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application [None req-64a8aced-fc1e-4323-9eeb-9cb251aa6d6e None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 15 10:19:26.475233 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:19:26.475233 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 15 10:19:26.475233 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 15 10:19:26.475233 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 15 10:19:26.475233 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 15 10:19:26.475233 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 15 10:19:26.475233 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 10:19:26.475233 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 15 10:19:26.475233 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application Jun 15 10:19:26.475233 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 15 10:19:26.475233 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application Jun 15 10:19:26.475233 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:19:26.475233 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:19:26.475233 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:19:26.475233 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:19:26.475233 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:19:26.476281 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:19:26.476281 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:19:26.476281 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:19:26.476281 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:19:26.476281 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:19:26.476281 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:19:26.476281 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 15 10:19:26.476281 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 15 10:19:26.476281 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 15 10:19:26.476281 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 15 10:19:26.476281 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:19:26.476281 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:19:26.476281 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:19:26.476281 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:19:26.476281 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:19:26.476281 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:19:26.477112 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:19:26.477112 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application with Lock( Jun 15 10:19:26.477112 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:19:26.477112 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:19:26.477112 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:19:26.477112 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:19:26.477112 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:19:26.477112 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:19:26.477112 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:19:26.477112 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:19:26.477112 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 15 10:19:26.477112 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 10:19:26.477112 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 15 10:19:26.477112 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application Jun 15 10:19:26.480084 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 458/918] 10.222.0.49 () {56 vars in 1114 bytes} [Wed Jun 15 10:19:26 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:19:26.487131 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335567cd-df37-48f3-9ba4-fb57e236e48d None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:19:26.506039 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95530a83-e6d2-4651-ba42-8295ae401f07 None None] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:26.506354 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95530a83-e6d2-4651-ba42-8295ae401f07 None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:26.506596 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95530a83-e6d2-4651-ba42-8295ae401f07 None None] PATH_INFO: `/` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:26.509163 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 459/919] 10.222.0.49 () {54 vars in 926 bytes} [Wed Jun 15 10:19:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 15 10:19:26.513659 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee726a5-e875-4e14-ad7e-f9c3283ba076 None None] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:26.513951 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee726a5-e875-4e14-ad7e-f9c3283ba076 None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:26.514230 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee726a5-e875-4e14-ad7e-f9c3283ba076 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:26.532997 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.auth.core [None req-3ee726a5-e875-4e14-ad7e-f9c3283ba076 None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:19:26.534206 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-3ee726a5-e875-4e14-ad7e-f9c3283ba076 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:19:26.590013 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:19:26.593707 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335567cd-df37-48f3-9ba4-fb57e236e48d None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:19:26.607118 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 460/920] 10.222.0.49 () {58 vars in 1023 bytes} [Wed Jun 15 10:19:26 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 15 10:19:26.629739 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee726a5-e875-4e14-ad7e-f9c3283ba076 None None] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:19:26.630309 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad908a40-9b30-40c5-a2cc-5cd56f1c8030 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:19:26.631195 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad908a40-9b30-40c5-a2cc-5cd56f1c8030 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:26.631850 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad908a40-9b30-40c5-a2cc-5cd56f1c8030 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:26.632270 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad908a40-9b30-40c5-a2cc-5cd56f1c8030 None admin] PATH_INFO: `/v3/services` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:26.633234 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad908a40-9b30-40c5-a2cc-5cd56f1c8030 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:19:26.633545 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad908a40-9b30-40c5-a2cc-5cd56f1c8030 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:19:26.635542 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad908a40-9b30-40c5-a2cc-5cd56f1c8030 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:19:26.641030 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 461/921] 10.222.0.49 () {56 vars in 1114 bytes} [Wed Jun 15 10:19:26 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:19:26.646620 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad908a40-9b30-40c5-a2cc-5cd56f1c8030 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:19:26.701633 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=100 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:19:26.743230 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee726a5-e875-4e14-ad7e-f9c3283ba076 None None] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:19:26.770356 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad908a40-9b30-40c5-a2cc-5cd56f1c8030 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:19:26.782629 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092b47d6-f1dc-4407-b79f-eb47b5cd4d05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:19:26.783337 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092b47d6-f1dc-4407-b79f-eb47b5cd4d05 None admin] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:26.783620 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092b47d6-f1dc-4407-b79f-eb47b5cd4d05 None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:26.783889 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092b47d6-f1dc-4407-b79f-eb47b5cd4d05 None admin] PATH_INFO: `/v3/services` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:26.784603 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-092b47d6-f1dc-4407-b79f-eb47b5cd4d05 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:19:26.786184 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-092b47d6-f1dc-4407-b79f-eb47b5cd4d05 None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:19:26.807454 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 461/922] 10.222.0.49 () {60 vars in 1141 bytes} [Wed Jun 15 10:19:26 2022] POST /identity/v3/services => generated 241 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 10:19:26.821233 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f620d8-be01-449d-9929-9dadd603281a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:19:26.822794 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f620d8-be01-449d-9929-9dadd603281a None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:26.823361 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f620d8-be01-449d-9929-9dadd603281a None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:26.823950 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f620d8-be01-449d-9929-9dadd603281a None admin] PATH_INFO: `/v3/services` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:26.824964 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f620d8-be01-449d-9929-9dadd603281a None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:19:26.825473 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f620d8-be01-449d-9929-9dadd603281a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:19:26.827167 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f620d8-be01-449d-9929-9dadd603281a None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:19:26.834030 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 462/923] 10.222.0.49 () {56 vars in 1114 bytes} [Wed Jun 15 10:19:26 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 10:19:26.840778 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092b47d6-f1dc-4407-b79f-eb47b5cd4d05 None admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:19:26.959996 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092b47d6-f1dc-4407-b79f-eb47b5cd4d05 None admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:19:26.985418 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:19:27.000616 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a7ebe6-c735-4a9f-8221-6c4574184b04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:19:27.001279 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a7ebe6-c735-4a9f-8221-6c4574184b04 None admin] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:27.001560 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a7ebe6-c735-4a9f-8221-6c4574184b04 None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:27.001831 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a7ebe6-c735-4a9f-8221-6c4574184b04 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:27.002569 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a7ebe6-c735-4a9f-8221-6c4574184b04 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:19:27.004003 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a7ebe6-c735-4a9f-8221-6c4574184b04 None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:19:27.027142 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 462/924] 10.222.0.49 () {60 vars in 1144 bytes} [Wed Jun 15 10:19:26 2022] POST /identity/v3/endpoints => generated 346 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 10:19:28.143514 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30dd3d7c-315e-4ffb-8dd1-c5f61e4b4d30 None None] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:28.144398 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30dd3d7c-315e-4ffb-8dd1-c5f61e4b4d30 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:28.144833 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30dd3d7c-315e-4ffb-8dd1-c5f61e4b4d30 None None] PATH_INFO: `/` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:28.146750 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 463/925] 10.222.0.49 () {54 vars in 926 bytes} [Wed Jun 15 10:19:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:19:28.153588 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f7de18-a924-4074-9af1-e22e65ee0ffd None None] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:28.154553 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f7de18-a924-4074-9af1-e22e65ee0ffd None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:28.155058 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f7de18-a924-4074-9af1-e22e65ee0ffd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:28.172575 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.auth.core [None req-83f7de18-a924-4074-9af1-e22e65ee0ffd None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:19:28.174097 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-83f7de18-a924-4074-9af1-e22e65ee0ffd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:19:28.249435 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 463/926] 10.222.0.49 () {58 vars in 1023 bytes} [Wed Jun 15 10:19:28 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 10:19:28.271975 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30dd3d7c-315e-4ffb-8dd1-c5f61e4b4d30 None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:19:28.298332 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:19:28.379248 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30dd3d7c-315e-4ffb-8dd1-c5f61e4b4d30 None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:19:28.416301 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502b6f21-753d-46c5-a4b1-d56cecbf0acb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:19:28.417044 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502b6f21-753d-46c5-a4b1-d56cecbf0acb None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:28.417328 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502b6f21-753d-46c5-a4b1-d56cecbf0acb None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:28.417601 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502b6f21-753d-46c5-a4b1-d56cecbf0acb None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:28.429206 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-502b6f21-753d-46c5-a4b1-d56cecbf0acb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 10:19:28.438216 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application [None req-502b6f21-753d-46c5-a4b1-d56cecbf0acb None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 15 10:19:28.438216 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:19:28.438216 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:19:28.438216 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:19:28.438216 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:19:28.438216 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:19:28.438216 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:19:28.438216 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:19:28.438216 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:19:28.438216 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:19:28.438216 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:19:28.438216 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:19:28.438216 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 10:19:28.438216 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 10:19:28.438216 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 10:19:28.438216 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 10:19:28.439245 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 10:19:28.439245 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 10:19:28.439245 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 10:19:28.439245 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 10:19:28.439245 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:19:28.439245 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:19:28.439245 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:19:28.439245 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:19:28.439245 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:19:28.439245 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:19:28.439245 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:19:28.439245 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application with Lock( Jun 15 10:19:28.439245 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:19:28.439245 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:19:28.439245 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:19:28.439245 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:19:28.440101 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:19:28.440101 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:19:28.440101 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:19:28.440101 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:19:28.440101 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 10:19:28.440101 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 10:19:28.440101 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 10:19:28.440101 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 10:19:28.440101 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 10:19:28.440101 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 10:19:28.440101 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 15 10:19:28.440101 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Jun 15 10:19:28.440101 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 464/927] 10.222.0.49 () {56 vars in 1120 bytes} [Wed Jun 15 10:19:28 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:19:28.448223 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f7de18-a924-4074-9af1-e22e65ee0ffd None None] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:19:28.524273 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6446848a-f788-454d-a085-96879112ed47 None None] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:28.524926 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6446848a-f788-454d-a085-96879112ed47 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:28.525779 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6446848a-f788-454d-a085-96879112ed47 None None] PATH_INFO: `/` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:28.531585 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 465/928] 10.222.0.49 () {54 vars in 926 bytes} [Wed Jun 15 10:19:28 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:19:28.533027 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3619547-13c6-4862-a71d-0aded371b3cd None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:28.533545 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3619547-13c6-4862-a71d-0aded371b3cd None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:28.534025 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3619547-13c6-4862-a71d-0aded371b3cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:28.552390 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-d3619547-13c6-4862-a71d-0aded371b3cd None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:19:28.553928 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-d3619547-13c6-4862-a71d-0aded371b3cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:19:28.559881 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f7de18-a924-4074-9af1-e22e65ee0ffd None None] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:19:28.562273 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:19:28.605263 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616f1d0b-891c-4064-988b-93be573ca69e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:19:28.605949 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616f1d0b-891c-4064-988b-93be573ca69e None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:28.606234 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616f1d0b-891c-4064-988b-93be573ca69e None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:28.606511 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616f1d0b-891c-4064-988b-93be573ca69e None admin] PATH_INFO: `/v3/roles` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:28.607286 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616f1d0b-891c-4064-988b-93be573ca69e None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:19:28.607987 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616f1d0b-891c-4064-988b-93be573ca69e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:19:28.609676 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616f1d0b-891c-4064-988b-93be573ca69e None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:19:28.625541 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 464/929] 10.222.0.49 () {56 vars in 1115 bytes} [Wed Jun 15 10:19:28 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:19:28.627683 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616f1d0b-891c-4064-988b-93be573ca69e None admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:19:28.643119 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 466/930] 10.222.0.49 () {58 vars in 1023 bytes} [Wed Jun 15 10:19:28 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 10:19:28.661451 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3619547-13c6-4862-a71d-0aded371b3cd None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:19:28.679590 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:19:28.746048 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616f1d0b-891c-4064-988b-93be573ca69e None admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:19:28.784315 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7209e986-1da5-4614-9c04-3ce5ca757668 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:19:28.784518 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3619547-13c6-4862-a71d-0aded371b3cd None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:19:28.785705 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7209e986-1da5-4614-9c04-3ce5ca757668 None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:28.786237 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7209e986-1da5-4614-9c04-3ce5ca757668 None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:28.786778 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7209e986-1da5-4614-9c04-3ce5ca757668 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:28.802325 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7209e986-1da5-4614-9c04-3ce5ca757668 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:19:28.804266 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7209e986-1da5-4614-9c04-3ce5ca757668 None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:19:28.822344 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a47829-64f3-4578-a653-1afa2c07fabf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:19:28.823168 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a47829-64f3-4578-a653-1afa2c07fabf None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:28.823450 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a47829-64f3-4578-a653-1afa2c07fabf None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:28.823724 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a47829-64f3-4578-a653-1afa2c07fabf None admin] PATH_INFO: `/v3/services/compute` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:28.824373 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a47829-64f3-4578-a653-1afa2c07fabf None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:19:28.825219 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application [None req-7209e986-1da5-4614-9c04-3ce5ca757668 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 10:19:28.825219 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:19:28.825219 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:19:28.825219 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:19:28.825219 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:19:28.825219 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:19:28.825219 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:19:28.825219 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:19:28.825219 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:19:28.825219 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:19:28.825219 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:19:28.825219 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:19:28.825219 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 10:19:28.825219 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 10:19:28.825219 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 10:19:28.826073 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 10:19:28.826073 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:19:28.826073 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:19:28.826073 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:19:28.826073 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:19:28.826073 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:19:28.826073 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:19:28.826073 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:19:28.826073 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application with Lock( Jun 15 10:19:28.826073 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:19:28.826073 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:19:28.826073 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:19:28.826073 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:19:28.826073 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:19:28.826073 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:19:28.826073 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:19:28.826073 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:19:28.826982 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 10:19:28.826982 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 10:19:28.826982 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 10:19:28.826982 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application Jun 15 10:19:28.827238 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a47829-64f3-4578-a653-1afa2c07fabf None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:19:28.833309 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 465/931] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Jun 15 10:19:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:19:28.835226 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7209e986-1da5-4614-9c04-3ce5ca757668 None admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:19:28.847863 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application [None req-62a47829-64f3-4578-a653-1afa2c07fabf None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jun 15 10:19:28.847863 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:19:28.847863 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 15 10:19:28.847863 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 15 10:19:28.847863 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 15 10:19:28.847863 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 15 10:19:28.847863 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 15 10:19:28.847863 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 10:19:28.847863 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 15 10:19:28.847863 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Jun 15 10:19:28.847863 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 15 10:19:28.847863 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Jun 15 10:19:28.847863 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:19:28.847863 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:19:28.847863 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:19:28.847863 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:19:28.847863 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:19:28.848904 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:19:28.848904 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:19:28.848904 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:19:28.848904 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:19:28.848904 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:19:28.848904 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:19:28.848904 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 15 10:19:28.848904 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 15 10:19:28.848904 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 15 10:19:28.848904 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 15 10:19:28.848904 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:19:28.848904 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:19:28.848904 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:19:28.848904 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:19:28.848904 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:19:28.848904 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:19:28.849795 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:19:28.849795 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application with Lock( Jun 15 10:19:28.849795 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:19:28.849795 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:19:28.849795 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:19:28.849795 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:19:28.849795 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:19:28.849795 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:19:28.849795 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:19:28.849795 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:19:28.849795 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 15 10:19:28.849795 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 10:19:28.849795 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 15 10:19:28.849795 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Jun 15 10:19:28.857828 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 467/932] 10.222.0.49 () {56 vars in 1111 bytes} [Wed Jun 15 10:19:28 2022] GET /identity/v3/services/compute => generated 88 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 10:19:28.859007 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=100 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:19:28.860168 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a47829-64f3-4578-a653-1afa2c07fabf None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:19:28.951762 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:19:28.960516 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7209e986-1da5-4614-9c04-3ce5ca757668 None admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:19:28.980656 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a47829-64f3-4578-a653-1afa2c07fabf None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:19:29.006221 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62970e2-b002-4025-86dd-d2774f037f3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:19:29.006891 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62970e2-b002-4025-86dd-d2774f037f3a None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:29.007197 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62970e2-b002-4025-86dd-d2774f037f3a None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:29.007472 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62970e2-b002-4025-86dd-d2774f037f3a None admin] PATH_INFO: `/v3/domains` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:29.008355 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d62970e2-b002-4025-86dd-d2774f037f3a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:19:29.008661 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d62970e2-b002-4025-86dd-d2774f037f3a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:19:29.011153 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d62970e2-b002-4025-86dd-d2774f037f3a None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:19:29.021398 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0782364a-ace5-4ca2-a9f6-7318b3208c7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:19:29.022439 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0782364a-ace5-4ca2-a9f6-7318b3208c7b None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:29.022970 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0782364a-ace5-4ca2-a9f6-7318b3208c7b None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:29.023544 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0782364a-ace5-4ca2-a9f6-7318b3208c7b None admin] PATH_INFO: `/v3/services` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:29.024523 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0782364a-ace5-4ca2-a9f6-7318b3208c7b None admin] RBAC: Adding query filter params (name=compute) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:19:29.029822 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 466/933] 10.222.0.49 () {56 vars in 1109 bytes} [Wed Jun 15 10:19:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:19:29.029962 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0782364a-ace5-4ca2-a9f6-7318b3208c7b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:19:29.029962 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0782364a-ace5-4ca2-a9f6-7318b3208c7b None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:19:29.032319 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6a59aa-62c0-4974-9111-4c4f49b108f5 None None] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:29.032611 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6a59aa-62c0-4974-9111-4c4f49b108f5 None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:29.032887 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6a59aa-62c0-4974-9111-4c4f49b108f5 None None] PATH_INFO: `/` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:29.034304 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 467/934] 10.222.0.49 () {54 vars in 926 bytes} [Wed Jun 15 10:19:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 15 10:19:29.034920 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6a59aa-62c0-4974-9111-4c4f49b108f5 None None] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:19:29.035398 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 468/935] 10.222.0.49 () {56 vars in 1112 bytes} [Wed Jun 15 10:19:28 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:19:29.040881 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a4daee-be63-478a-bb7d-5b26adbf99e1 None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:29.041217 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a4daee-be63-478a-bb7d-5b26adbf99e1 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:29.041455 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a4daee-be63-478a-bb7d-5b26adbf99e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:29.059528 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-18a4daee-be63-478a-bb7d-5b26adbf99e1 None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:19:29.060718 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-18a4daee-be63-478a-bb7d-5b26adbf99e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:19:29.124171 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=100 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:19:29.136293 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 469/936] 10.222.0.49 () {58 vars in 1023 bytes} [Wed Jun 15 10:19:29 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 10:19:29.143970 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a4daee-be63-478a-bb7d-5b26adbf99e1 None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:19:29.154077 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6a59aa-62c0-4974-9111-4c4f49b108f5 None None] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:19:29.196467 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa630466-aefa-487b-bdaa-5d1d9274bfde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:19:29.198232 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa630466-aefa-487b-bdaa-5d1d9274bfde None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:29.198745 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa630466-aefa-487b-bdaa-5d1d9274bfde None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:29.199285 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa630466-aefa-487b-bdaa-5d1d9274bfde None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:29.206992 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa630466-aefa-487b-bdaa-5d1d9274bfde None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:19:29.208603 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa630466-aefa-487b-bdaa-5d1d9274bfde None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:19:29.218679 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:19:29.224037 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application [None req-aa630466-aefa-487b-bdaa-5d1d9274bfde None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 15 10:19:29.224037 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:19:29.224037 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:19:29.224037 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:19:29.224037 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:19:29.224037 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:19:29.224037 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:19:29.224037 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:19:29.224037 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:19:29.224037 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:19:29.224037 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:19:29.224037 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:19:29.224037 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 15 10:19:29.224037 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 15 10:19:29.224037 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 15 10:19:29.224037 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 15 10:19:29.224991 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:19:29.224991 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:19:29.224991 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 15 10:19:29.224991 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 10:19:29.224991 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 15 10:19:29.224991 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 10:19:29.224991 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:19:29.224991 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:19:29.224991 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:19:29.224991 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:19:29.224991 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:19:29.224991 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application with Lock( Jun 15 10:19:29.224991 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:19:29.224991 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:19:29.224991 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:19:29.224991 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:19:29.224991 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:19:29.224991 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:19:29.225932 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:19:29.225932 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:19:29.225932 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 15 10:19:29.225932 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 15 10:19:29.225932 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 15 10:19:29.225932 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 15 10:19:29.225932 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 15 10:19:29.225932 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 15 10:19:29.225932 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 15 10:19:29.225932 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application Jun 15 10:19:29.225932 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 468/937] 10.222.0.49 () {56 vars in 1117 bytes} [Wed Jun 15 10:19:29 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:19:29.226521 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa630466-aefa-487b-bdaa-5d1d9274bfde None admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:19:29.255902 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a4daee-be63-478a-bb7d-5b26adbf99e1 None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:19:29.288982 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675f56a7-1eac-43c4-9354-32f1deb91acb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:19:29.289646 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675f56a7-1eac-43c4-9354-32f1deb91acb None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:29.289931 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675f56a7-1eac-43c4-9354-32f1deb91acb None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:29.290210 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675f56a7-1eac-43c4-9354-32f1deb91acb None admin] PATH_INFO: `/v3/services` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:29.290949 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-675f56a7-1eac-43c4-9354-32f1deb91acb None admin] RBAC: Adding query filter params (type=compute) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:19:29.291287 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-675f56a7-1eac-43c4-9354-32f1deb91acb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:19:29.292684 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-675f56a7-1eac-43c4-9354-32f1deb91acb None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:19:29.297124 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 470/938] 10.222.0.49 () {56 vars in 1112 bytes} [Wed Jun 15 10:19:29 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:19:29.298301 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846d002f-3709-4175-b9d7-840f3d6e4ffc None None] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:29.298590 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846d002f-3709-4175-b9d7-840f3d6e4ffc None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:29.298866 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846d002f-3709-4175-b9d7-840f3d6e4ffc None None] PATH_INFO: `/` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:29.300396 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 471/939] 10.222.0.49 () {54 vars in 926 bytes} [Wed Jun 15 10:19:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:19:29.301117 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846d002f-3709-4175-b9d7-840f3d6e4ffc None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:19:29.342119 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa630466-aefa-487b-bdaa-5d1d9274bfde None admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:19:29.382025 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dcece32-bf07-4008-ad2b-5902aabc0835 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:19:29.382860 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcece32-bf07-4008-ad2b-5902aabc0835 None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:29.383176 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcece32-bf07-4008-ad2b-5902aabc0835 None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:29.383456 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcece32-bf07-4008-ad2b-5902aabc0835 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:29.384128 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dcece32-bf07-4008-ad2b-5902aabc0835 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:19:29.386718 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dcece32-bf07-4008-ad2b-5902aabc0835 None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:19:29.400373 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application [None req-9dcece32-bf07-4008-ad2b-5902aabc0835 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 15 10:19:29.400373 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:19:29.400373 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 15 10:19:29.400373 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 15 10:19:29.400373 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 15 10:19:29.400373 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 15 10:19:29.400373 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 15 10:19:29.400373 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 10:19:29.400373 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 15 10:19:29.400373 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application Jun 15 10:19:29.400373 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 15 10:19:29.400373 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application Jun 15 10:19:29.400373 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:19:29.400373 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:19:29.400373 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:19:29.400373 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:19:29.400373 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:19:29.401442 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:19:29.401442 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:19:29.401442 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:19:29.401442 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:19:29.401442 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:19:29.401442 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:19:29.401442 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 15 10:19:29.401442 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 15 10:19:29.401442 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 15 10:19:29.401442 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 15 10:19:29.401442 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:19:29.401442 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:19:29.401442 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:19:29.401442 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:19:29.401442 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:19:29.401442 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:19:29.402319 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:19:29.402319 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application with Lock( Jun 15 10:19:29.402319 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:19:29.402319 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:19:29.402319 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:19:29.402319 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:19:29.402319 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:19:29.402319 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:19:29.402319 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:19:29.402319 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:19:29.402319 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 15 10:19:29.402319 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 10:19:29.402319 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 15 10:19:29.402319 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application Jun 15 10:19:29.402319 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 469/940] 10.222.0.49 () {56 vars in 1126 bytes} [Wed Jun 15 10:19:29 2022] GET /identity/v3/services/object-store => generated 93 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:19:29.403177 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f0b6bc-3106-4a26-a225-ece755a04f1b None None] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:29.403794 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f0b6bc-3106-4a26-a225-ece755a04f1b None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:29.403794 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f0b6bc-3106-4a26-a225-ece755a04f1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:29.408994 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=100 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:19:29.421264 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846d002f-3709-4175-b9d7-840f3d6e4ffc None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:19:29.423337 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.auth.core [None req-84f0b6bc-3106-4a26-a225-ece755a04f1b None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:19:29.425790 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-84f0b6bc-3106-4a26-a225-ece755a04f1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:19:29.460363 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9e879e-a607-4859-8102-86de94c13e85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:19:29.460975 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9e879e-a607-4859-8102-86de94c13e85 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:29.461260 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9e879e-a607-4859-8102-86de94c13e85 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:29.461538 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9e879e-a607-4859-8102-86de94c13e85 None admin] PATH_INFO: `/v3/users` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:29.462410 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a9e879e-a607-4859-8102-86de94c13e85 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:19:29.462723 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a9e879e-a607-4859-8102-86de94c13e85 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:19:29.464218 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a9e879e-a607-4859-8102-86de94c13e85 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:19:29.468140 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:19:29.483438 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 472/941] 10.222.0.49 () {56 vars in 1149 bytes} [Wed Jun 15 10:19:29 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:19:29.484273 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9e879e-a607-4859-8102-86de94c13e85 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:19:29.505126 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 470/942] 10.222.0.49 () {58 vars in 1023 bytes} [Wed Jun 15 10:19:29 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 10:19:29.506350 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f0b6bc-3106-4a26-a225-ece755a04f1b None None] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:19:29.594639 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9e879e-a607-4859-8102-86de94c13e85 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:19:29.617523 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f0b6bc-3106-4a26-a225-ece755a04f1b None None] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:19:29.630879 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c50d945-27eb-429f-af2e-e7390adca8a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:19:29.631583 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c50d945-27eb-429f-af2e-e7390adca8a7 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:29.631872 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c50d945-27eb-429f-af2e-e7390adca8a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:29.632133 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c50d945-27eb-429f-af2e-e7390adca8a7 None admin] PATH_INFO: `/v3/services` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:29.632833 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c50d945-27eb-429f-af2e-e7390adca8a7 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:19:29.633313 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c50d945-27eb-429f-af2e-e7390adca8a7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:19:29.634512 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c50d945-27eb-429f-af2e-e7390adca8a7 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:19:29.639504 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 473/943] 10.222.0.49 () {56 vars in 1122 bytes} [Wed Jun 15 10:19:29 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:19:29.640178 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c50d945-27eb-429f-af2e-e7390adca8a7 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:19:29.655061 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=100 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:19:29.660983 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827bdf6d-484c-4f53-bc6a-740f824b5b0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:19:29.661783 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827bdf6d-484c-4f53-bc6a-740f824b5b0a None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:29.662068 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827bdf6d-484c-4f53-bc6a-740f824b5b0a None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:29.662340 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827bdf6d-484c-4f53-bc6a-740f824b5b0a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:29.675116 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-827bdf6d-484c-4f53-bc6a-740f824b5b0a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:19:29.677094 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-827bdf6d-484c-4f53-bc6a-740f824b5b0a None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:19:29.700998 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application [None req-827bdf6d-484c-4f53-bc6a-740f824b5b0a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 10:19:29.700998 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:19:29.700998 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:19:29.700998 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:19:29.700998 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:19:29.700998 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:19:29.700998 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:19:29.700998 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:19:29.700998 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:19:29.700998 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:19:29.700998 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:19:29.700998 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:19:29.700998 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 10:19:29.700998 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 10:19:29.700998 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 10:19:29.701913 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 10:19:29.701913 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:19:29.701913 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:19:29.701913 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:19:29.701913 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:19:29.701913 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:19:29.701913 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:19:29.701913 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:19:29.701913 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application with Lock( Jun 15 10:19:29.701913 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:19:29.701913 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:19:29.701913 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:19:29.701913 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:19:29.701913 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:19:29.701913 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:19:29.701913 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:19:29.701913 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:19:29.703252 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 10:19:29.703252 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 10:19:29.703252 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 10:19:29.703252 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application Jun 15 10:19:29.703643 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:19:29.703734 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 471/944] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Jun 15 10:19:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:19:29.703734 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827bdf6d-484c-4f53-bc6a-740f824b5b0a None admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:19:29.749124 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c50d945-27eb-429f-af2e-e7390adca8a7 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:19:29.785040 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07664bf-1f55-47e5-aac3-15dcfc09bfac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:19:29.785726 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07664bf-1f55-47e5-aac3-15dcfc09bfac None admin] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:29.785987 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07664bf-1f55-47e5-aac3-15dcfc09bfac None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:29.786263 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07664bf-1f55-47e5-aac3-15dcfc09bfac None admin] PATH_INFO: `/v3/projects` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:29.786962 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07664bf-1f55-47e5-aac3-15dcfc09bfac None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:19:29.788440 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07664bf-1f55-47e5-aac3-15dcfc09bfac None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:19:29.789809 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: WARNING keystone.server.flask.common [None req-b07664bf-1f55-47e5-aac3-15dcfc09bfac None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Jun 15 10:19:29.807845 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.sql.core [None req-b07664bf-1f55-47e5-aac3-15dcfc09bfac None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jun 15 10:19:29.807845 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jun 15 10:19:29.807845 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [parameters: {'id': 'f90c9bda5d42457b9b9b1ebe4b61a425', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jun 15 10:19:29.807845 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80120) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jun 15 10:19:29.815283 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application [None req-b07664bf-1f55-47e5-aac3-15dcfc09bfac None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id f90c9bda5d42457b9b9b1ebe4b61a425.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id f90c9bda5d42457b9b9b1ebe4b61a425. Jun 15 10:19:29.815283 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:19:29.815283 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 15 10:19:29.815283 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 15 10:19:29.815283 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 15 10:19:29.815283 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 15 10:19:29.815283 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 15 10:19:29.815283 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application result = self._query(query) Jun 15 10:19:29.815283 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 15 10:19:29.815283 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application conn.query(q) Jun 15 10:19:29.815283 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 15 10:19:29.815283 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 15 10:19:29.815283 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 15 10:19:29.816112 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application result.read() Jun 15 10:19:29.816112 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 15 10:19:29.816112 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 15 10:19:29.816112 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 15 10:19:29.816112 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 15 10:19:29.816112 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 15 10:19:29.816112 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 15 10:19:29.816112 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 15 10:19:29.816112 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 15 10:19:29.816112 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jun 15 10:19:29.816112 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Jun 15 10:19:29.816112 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jun 15 10:19:29.816112 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Jun 15 10:19:29.816112 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:19:29.816112 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jun 15 10:19:29.816112 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jun 15 10:19:29.816112 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Jun 15 10:19:29.816112 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return project_ref.to_dict() Jun 15 10:19:29.817137 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 15 10:19:29.817137 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application next(self.gen) Jun 15 10:19:29.817137 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jun 15 10:19:29.817137 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application yield resource Jun 15 10:19:29.817137 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 15 10:19:29.817137 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application next(self.gen) Jun 15 10:19:29.817137 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jun 15 10:19:29.817137 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application self.session.rollback() Jun 15 10:19:29.817137 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jun 15 10:19:29.817137 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application self.force_reraise() Jun 15 10:19:29.817137 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jun 15 10:19:29.817137 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application raise self.value Jun 15 10:19:29.817137 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jun 15 10:19:29.817137 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jun 15 10:19:29.817137 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jun 15 10:19:29.817137 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application session.commit() Jun 15 10:19:29.817137 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jun 15 10:19:29.818043 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jun 15 10:19:29.818043 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jun 15 10:19:29.818043 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application self._prepare_impl() Jun 15 10:19:29.818043 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jun 15 10:19:29.818043 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application self.session.flush() Jun 15 10:19:29.818043 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jun 15 10:19:29.818043 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application self._flush(objects) Jun 15 10:19:29.818043 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jun 15 10:19:29.818043 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jun 15 10:19:29.818043 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jun 15 10:19:29.818043 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application compat.raise_( Jun 15 10:19:29.818043 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 15 10:19:29.818043 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application raise exception Jun 15 10:19:29.818043 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jun 15 10:19:29.818043 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application flush_context.execute() Jun 15 10:19:29.818043 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jun 15 10:19:29.818043 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application rec.execute(self) Jun 15 10:19:29.820032 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jun 15 10:19:29.820032 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jun 15 10:19:29.820032 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jun 15 10:19:29.820032 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application _emit_insert_statements( Jun 15 10:19:29.820032 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jun 15 10:19:29.820032 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application c = connection._execute_20( Jun 15 10:19:29.820032 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jun 15 10:19:29.820032 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jun 15 10:19:29.820032 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection Jun 15 10:19:29.820032 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jun 15 10:19:29.820032 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jun 15 10:19:29.820032 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application ret = self._execute_context( Jun 15 10:19:29.820032 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jun 15 10:19:29.820032 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jun 15 10:19:29.820032 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jun 15 10:19:29.822977 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jun 15 10:19:29.822977 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 15 10:19:29.822977 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application raise exception Jun 15 10:19:29.822977 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 15 10:19:29.822977 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 15 10:19:29.822977 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 15 10:19:29.822977 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 15 10:19:29.822977 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 15 10:19:29.822977 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application result = self._query(query) Jun 15 10:19:29.822977 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 15 10:19:29.822977 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application conn.query(q) Jun 15 10:19:29.822977 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 15 10:19:29.822977 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 15 10:19:29.822977 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 15 10:19:29.822977 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application result.read() Jun 15 10:19:29.822977 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 15 10:19:29.822977 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 15 10:19:29.824215 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 15 10:19:29.824215 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 15 10:19:29.824215 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 15 10:19:29.824215 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 15 10:19:29.824215 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 15 10:19:29.824215 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 15 10:19:29.824215 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jun 15 10:19:29.824215 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jun 15 10:19:29.824215 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application [parameters: {'id': 'f90c9bda5d42457b9b9b1ebe4b61a425', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jun 15 10:19:29.824215 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jun 15 10:19:29.824215 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Jun 15 10:19:29.824215 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 15 10:19:29.824215 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Jun 15 10:19:29.824215 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:19:29.824215 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Jun 15 10:19:29.825219 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Jun 15 10:19:29.825219 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jun 15 10:19:29.825219 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jun 15 10:19:29.825219 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Jun 15 10:19:29.825219 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Jun 15 10:19:29.825219 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 15 10:19:29.825219 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Jun 15 10:19:29.825219 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:19:29.825219 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:19:29.825219 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:19:29.825219 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:19:29.825219 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:19:29.825219 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:19:29.825219 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:19:29.825219 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:19:29.825219 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:19:29.825219 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:19:29.826332 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:19:29.826332 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Jun 15 10:19:29.826332 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Jun 15 10:19:29.826332 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:19:29.826332 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:19:29.826332 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Jun 15 10:19:29.826332 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application raise exception.Conflict( Jun 15 10:19:29.826332 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id f90c9bda5d42457b9b9b1ebe4b61a425. Jun 15 10:19:29.826332 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Jun 15 10:19:29.826332 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 474/945] 10.222.0.49 () {60 vars in 1140 bytes} [Wed Jun 15 10:19:29 2022] POST /identity/v3/projects => generated 262 bytes in 177 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:19:29.826332 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07664bf-1f55-47e5-aac3-15dcfc09bfac None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:19:29.827128 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827bdf6d-484c-4f53-bc6a-740f824b5b0a None admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:19:29.862571 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183a8bb2-07c1-4643-ba9d-2c1b24fdf99f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:19:29.863367 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183a8bb2-07c1-4643-ba9d-2c1b24fdf99f None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:29.863757 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183a8bb2-07c1-4643-ba9d-2c1b24fdf99f None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:29.864121 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183a8bb2-07c1-4643-ba9d-2c1b24fdf99f None admin] PATH_INFO: `/v3/services` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:29.864829 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183a8bb2-07c1-4643-ba9d-2c1b24fdf99f None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:19:29.865133 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183a8bb2-07c1-4643-ba9d-2c1b24fdf99f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:19:29.866504 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183a8bb2-07c1-4643-ba9d-2c1b24fdf99f None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:19:29.871593 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 472/946] 10.222.0.49 () {56 vars in 1122 bytes} [Wed Jun 15 10:19:29 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:19:29.872264 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183a8bb2-07c1-4643-ba9d-2c1b24fdf99f None admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:19:29.923733 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=100 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:19:29.928423 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07664bf-1f55-47e5-aac3-15dcfc09bfac None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:19:29.942679 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:19:29.964884 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16adc21a-4238-4e43-822f-fed742f9c94c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:19:29.965554 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16adc21a-4238-4e43-822f-fed742f9c94c None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:29.965841 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16adc21a-4238-4e43-822f-fed742f9c94c None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:29.966122 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16adc21a-4238-4e43-822f-fed742f9c94c None admin] PATH_INFO: `/v3/domains` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:29.966844 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16adc21a-4238-4e43-822f-fed742f9c94c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:19:29.967185 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16adc21a-4238-4e43-822f-fed742f9c94c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:19:29.968749 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16adc21a-4238-4e43-822f-fed742f9c94c None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:19:29.982483 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183a8bb2-07c1-4643-ba9d-2c1b24fdf99f None admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:19:29.984232 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 475/947] 10.222.0.49 () {56 vars in 1109 bytes} [Wed Jun 15 10:19:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:19:29.984914 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16adc21a-4238-4e43-822f-fed742f9c94c None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:19:30.024744 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797ca5db-6b3a-43d5-b9ee-d7747d2601ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:19:30.025998 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797ca5db-6b3a-43d5-b9ee-d7747d2601ca None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:30.026284 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797ca5db-6b3a-43d5-b9ee-d7747d2601ca None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:30.026562 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797ca5db-6b3a-43d5-b9ee-d7747d2601ca None admin] PATH_INFO: `/v3/projects/service` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:30.033544 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-797ca5db-6b3a-43d5-b9ee-d7747d2601ca None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:19:30.035012 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-797ca5db-6b3a-43d5-b9ee-d7747d2601ca None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:19:30.049971 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application [None req-797ca5db-6b3a-43d5-b9ee-d7747d2601ca None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 10:19:30.049971 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:19:30.049971 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:19:30.049971 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:19:30.049971 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:19:30.049971 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:19:30.049971 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:19:30.049971 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:19:30.049971 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:19:30.049971 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:19:30.049971 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:19:30.049971 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:19:30.049971 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 15 10:19:30.049971 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 15 10:19:30.049971 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 15 10:19:30.050884 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 15 10:19:30.050884 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:19:30.050884 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:19:30.050884 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:19:30.050884 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:19:30.050884 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:19:30.050884 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:19:30.050884 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:19:30.050884 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application with Lock( Jun 15 10:19:30.050884 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:19:30.050884 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:19:30.050884 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:19:30.050884 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:19:30.050884 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:19:30.050884 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:19:30.050884 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:19:30.050884 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:19:30.052256 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 15 10:19:30.052256 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 15 10:19:30.052256 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 15 10:19:30.052256 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 15 10:19:30.052256 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 15 10:19:30.052256 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 15 10:19:30.052256 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 10:19:30.052256 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application Jun 15 10:19:30.052256 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 473/948] 10.222.0.49 () {56 vars in 1111 bytes} [Wed Jun 15 10:19:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:19:30.052256 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797ca5db-6b3a-43d5-b9ee-d7747d2601ca None admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:19:30.098469 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16adc21a-4238-4e43-822f-fed742f9c94c None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:19:30.138234 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59bbc9fd-a7b0-4d79-afc8-0fe56c71590a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:19:30.139197 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bbc9fd-a7b0-4d79-afc8-0fe56c71590a None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:30.139506 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bbc9fd-a7b0-4d79-afc8-0fe56c71590a None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:30.139932 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bbc9fd-a7b0-4d79-afc8-0fe56c71590a None admin] PATH_INFO: `/v3/endpoints` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:30.140851 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59bbc9fd-a7b0-4d79-afc8-0fe56c71590a None admin] RBAC: Adding query filter params (interface=public, service_id=6d4c1066aada4da5851a15eb90fa1748, region_id=RegionOne) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:19:30.141162 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59bbc9fd-a7b0-4d79-afc8-0fe56c71590a None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:19:30.143058 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59bbc9fd-a7b0-4d79-afc8-0fe56c71590a None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:19:30.148432 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 476/949] 10.222.0.49 () {56 vars in 1251 bytes} [Wed Jun 15 10:19:29 2022] GET /identity/v3/endpoints?service_id=6d4c1066aada4da5851a15eb90fa1748&interface=public®ion_id=RegionOne => generated 194 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:19:30.151943 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59bbc9fd-a7b0-4d79-afc8-0fe56c71590a None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:19:30.163486 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797ca5db-6b3a-43d5-b9ee-d7747d2601ca None admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:19:30.173021 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=100 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:19:30.193650 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:19:30.202211 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53dc4c2a-9212-4275-a573-0940c94141b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:19:30.203198 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53dc4c2a-9212-4275-a573-0940c94141b1 None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:30.203737 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53dc4c2a-9212-4275-a573-0940c94141b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:30.204294 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53dc4c2a-9212-4275-a573-0940c94141b1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:30.211143 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53dc4c2a-9212-4275-a573-0940c94141b1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:19:30.213245 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53dc4c2a-9212-4275-a573-0940c94141b1 None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:19:30.228691 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application [None req-53dc4c2a-9212-4275-a573-0940c94141b1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 10:19:30.228691 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:19:30.228691 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:19:30.228691 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:19:30.228691 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:19:30.228691 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:19:30.228691 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:19:30.228691 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:19:30.228691 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:19:30.228691 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:19:30.228691 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:19:30.228691 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:19:30.228691 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 15 10:19:30.228691 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 15 10:19:30.228691 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 15 10:19:30.229804 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 15 10:19:30.229804 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:19:30.229804 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:19:30.229804 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:19:30.229804 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:19:30.229804 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:19:30.229804 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:19:30.229804 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:19:30.229804 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application with Lock( Jun 15 10:19:30.229804 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:19:30.229804 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:19:30.229804 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:19:30.229804 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:19:30.229804 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:19:30.229804 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:19:30.229804 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:19:30.229804 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:19:30.235432 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 15 10:19:30.235432 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 15 10:19:30.235432 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 15 10:19:30.235432 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 15 10:19:30.235432 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 15 10:19:30.235432 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 15 10:19:30.235432 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 10:19:30.235432 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application Jun 15 10:19:30.235432 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 474/950] 10.222.0.49 () {56 vars in 1111 bytes} [Wed Jun 15 10:19:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:19:30.235432 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53dc4c2a-9212-4275-a573-0940c94141b1 None admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:19:30.262085 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59bbc9fd-a7b0-4d79-afc8-0fe56c71590a None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:19:30.296864 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e8b826-f13c-41c7-a637-7f39426bce98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:19:30.297558 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e8b826-f13c-41c7-a637-7f39426bce98 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:30.297846 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e8b826-f13c-41c7-a637-7f39426bce98 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:30.298127 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e8b826-f13c-41c7-a637-7f39426bce98 None admin] PATH_INFO: `/v3/projects` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:30.298874 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e8b826-f13c-41c7-a637-7f39426bce98 None admin] RBAC: Adding query filter params (name=service) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:19:30.299300 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e8b826-f13c-41c7-a637-7f39426bce98 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:19:30.300981 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e8b826-f13c-41c7-a637-7f39426bce98 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:19:30.327121 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 477/951] 10.222.0.49 () {56 vars in 1112 bytes} [Wed Jun 15 10:19:30 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:19:30.327121 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e8b826-f13c-41c7-a637-7f39426bce98 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:19:30.348442 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53dc4c2a-9212-4275-a573-0940c94141b1 None admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:19:30.405349 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e664b7c5-1b65-436f-89ef-a377251ae73f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:19:30.406051 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e664b7c5-1b65-436f-89ef-a377251ae73f None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:30.406337 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e664b7c5-1b65-436f-89ef-a377251ae73f None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:30.406614 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e664b7c5-1b65-436f-89ef-a377251ae73f None admin] PATH_INFO: `/v3/services` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:30.407390 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e664b7c5-1b65-436f-89ef-a377251ae73f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:19:30.408768 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e664b7c5-1b65-436f-89ef-a377251ae73f None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:19:30.416108 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 475/952] 10.222.0.49 () {56 vars in 1087 bytes} [Wed Jun 15 10:19:30 2022] GET /identity/v3/services => generated 1497 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 10:19:30.450623 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:19:30.455318 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e8b826-f13c-41c7-a637-7f39426bce98 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:19:30.498698 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96399238-a5de-4ae9-adfc-b16ac81b634f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:19:30.499425 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96399238-a5de-4ae9-adfc-b16ac81b634f None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:30.499715 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96399238-a5de-4ae9-adfc-b16ac81b634f None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:30.499996 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96399238-a5de-4ae9-adfc-b16ac81b634f None admin] PATH_INFO: `/v3/projects` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:30.500829 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96399238-a5de-4ae9-adfc-b16ac81b634f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:19:30.501139 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96399238-a5de-4ae9-adfc-b16ac81b634f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:19:30.502643 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96399238-a5de-4ae9-adfc-b16ac81b634f None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:19:30.517312 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 478/953] 10.222.0.49 () {56 vars in 1148 bytes} [Wed Jun 15 10:19:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:19:30.523379 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e664b7c5-1b65-436f-89ef-a377251ae73f None admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:19:30.561029 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=100 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:19:30.638874 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e664b7c5-1b65-436f-89ef-a377251ae73f None admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:19:30.675260 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3816e63-2519-43c6-970c-af641e71e76f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:19:30.676514 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3816e63-2519-43c6-970c-af641e71e76f None admin] REQUEST_METHOD: `PUT` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:30.677000 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3816e63-2519-43c6-970c-af641e71e76f None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:30.677479 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3816e63-2519-43c6-970c-af641e71e76f None admin] PATH_INFO: `/v3/projects/559b3e713f5844b6ae37151d56dd419f/users/103e10244eca444b86b9c51ff314d445/roles/d962cdb5fdb24466a560f5d7bcb1e034` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:30.680075 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3816e63-2519-43c6-970c-af641e71e76f None admin] RBAC: Authorizing `identity:create_grant(project_id=559b3e713f5844b6ae37151d56dd419f, user_id=103e10244eca444b86b9c51ff314d445, role_id=d962cdb5fdb24466a560f5d7bcb1e034)` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:19:30.682239 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3816e63-2519-43c6-970c-af641e71e76f None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:19:30.699887 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 476/954] 10.222.0.49 () {58 vars in 1439 bytes} [Wed Jun 15 10:19:30 2022] PUT /identity/v3/projects/559b3e713f5844b6ae37151d56dd419f/users/103e10244eca444b86b9c51ff314d445/roles/d962cdb5fdb24466a560f5d7bcb1e034 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:19:31.463209 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463798cd-ef0b-40c8-a2e9-7f9d1054e1de None None] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:31.463760 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463798cd-ef0b-40c8-a2e9-7f9d1054e1de None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:31.463760 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463798cd-ef0b-40c8-a2e9-7f9d1054e1de None None] PATH_INFO: `/` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:31.465376 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 479/955] 10.222.0.49 () {54 vars in 926 bytes} [Wed Jun 15 10:19:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:19:31.470550 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513a6572-4901-481b-9115-ac70d9928bfa None None] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:31.470944 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513a6572-4901-481b-9115-ac70d9928bfa None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:31.471153 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513a6572-4901-481b-9115-ac70d9928bfa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:31.488742 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.auth.core [None req-513a6572-4901-481b-9115-ac70d9928bfa None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:19:31.489948 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-513a6572-4901-481b-9115-ac70d9928bfa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:19:31.568347 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 477/956] 10.222.0.49 () {58 vars in 1023 bytes} [Wed Jun 15 10:19:31 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 10:19:31.591936 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463798cd-ef0b-40c8-a2e9-7f9d1054e1de None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:19:31.694552 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463798cd-ef0b-40c8-a2e9-7f9d1054e1de None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:19:31.728960 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a492eb4-809f-40ff-81c6-6c7275c78a7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:19:31.729553 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a492eb4-809f-40ff-81c6-6c7275c78a7f None admin] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:31.729832 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a492eb4-809f-40ff-81c6-6c7275c78a7f None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:31.730104 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a492eb4-809f-40ff-81c6-6c7275c78a7f None admin] PATH_INFO: `/v3/services` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:31.730815 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a492eb4-809f-40ff-81c6-6c7275c78a7f None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:19:31.732191 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a492eb4-809f-40ff-81c6-6c7275c78a7f None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:19:31.749904 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 480/957] 10.222.0.49 () {60 vars in 1141 bytes} [Wed Jun 15 10:19:31 2022] POST /identity/v3/services => generated 242 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 10:19:32.592775 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749accb3-ff8d-43de-98c6-89dc5400755c None None] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:32.593849 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749accb3-ff8d-43de-98c6-89dc5400755c None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:32.594355 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749accb3-ff8d-43de-98c6-89dc5400755c None None] PATH_INFO: `/` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:32.596385 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 478/958] 10.222.0.49 () {54 vars in 926 bytes} [Wed Jun 15 10:19:32 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:19:32.602708 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2a3b0e-ccde-47b1-81b5-0d0d335f45a4 None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:32.603739 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2a3b0e-ccde-47b1-81b5-0d0d335f45a4 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:32.604255 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2a3b0e-ccde-47b1-81b5-0d0d335f45a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:32.625636 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-bb2a3b0e-ccde-47b1-81b5-0d0d335f45a4 None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:19:32.627171 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-bb2a3b0e-ccde-47b1-81b5-0d0d335f45a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:19:32.641050 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:19:32.647049 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f308dc8a-a911-4ad4-b3c8-e965855fe05d None None] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:32.647374 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f308dc8a-a911-4ad4-b3c8-e965855fe05d None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:32.650412 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f308dc8a-a911-4ad4-b3c8-e965855fe05d None None] PATH_INFO: `/` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:32.650412 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 479/959] 10.222.0.49 () {54 vars in 926 bytes} [Wed Jun 15 10:19:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:19:32.654869 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c69104c-478f-48e9-9a51-f7da86ccb211 None None] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:32.655199 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c69104c-478f-48e9-9a51-f7da86ccb211 None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:32.655487 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c69104c-478f-48e9-9a51-f7da86ccb211 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:32.673546 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.auth.core [None req-6c69104c-478f-48e9-9a51-f7da86ccb211 None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:19:32.675028 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-6c69104c-478f-48e9-9a51-f7da86ccb211 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:19:32.696312 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:19:32.715232 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 481/960] 10.222.0.49 () {58 vars in 1023 bytes} [Wed Jun 15 10:19:32 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 10:19:32.741979 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2a3b0e-ccde-47b1-81b5-0d0d335f45a4 None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:19:32.766377 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 480/961] 10.222.0.49 () {58 vars in 1023 bytes} [Wed Jun 15 10:19:32 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 15 10:19:32.800975 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c69104c-478f-48e9-9a51-f7da86ccb211 None None] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:19:32.859329 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2a3b0e-ccde-47b1-81b5-0d0d335f45a4 None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:19:32.896431 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92057c1-318f-46f6-a851-e7a2df7708d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:19:32.897118 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92057c1-318f-46f6-a851-e7a2df7708d4 None admin] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:32.897405 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92057c1-318f-46f6-a851-e7a2df7708d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:32.897684 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92057c1-318f-46f6-a851-e7a2df7708d4 None admin] PATH_INFO: `/v3/users` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:32.898397 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a92057c1-318f-46f6-a851-e7a2df7708d4 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:19:32.899883 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a92057c1-318f-46f6-a851-e7a2df7708d4 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:19:32.911405 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c69104c-478f-48e9-9a51-f7da86ccb211 None None] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:19:32.912779 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.sql.core [None req-a92057c1-318f-46f6-a851-e7a2df7708d4 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jun 15 10:19:32.912779 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jun 15 10:19:32.912779 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [parameters: {'user_id': '72f93a178f4642fcb940c3e2614bbd70', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jun 15 10:19:32.912779 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80120) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jun 15 10:19:32.920103 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application [None req-a92057c1-318f-46f6-a851-e7a2df7708d4 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jun 15 10:19:32.920103 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:19:32.920103 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 15 10:19:32.920103 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 15 10:19:32.920103 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 15 10:19:32.920103 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 15 10:19:32.920103 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 15 10:19:32.920103 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application result = self._query(query) Jun 15 10:19:32.920103 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 15 10:19:32.920103 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application conn.query(q) Jun 15 10:19:32.920103 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 15 10:19:32.920103 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 15 10:19:32.920103 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 15 10:19:32.920103 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application result.read() Jun 15 10:19:32.920103 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 15 10:19:32.920989 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 15 10:19:32.920989 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 15 10:19:32.920989 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 15 10:19:32.920989 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 15 10:19:32.920989 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 15 10:19:32.920989 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 15 10:19:32.920989 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 15 10:19:32.920989 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jun 15 10:19:32.920989 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Jun 15 10:19:32.920989 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jun 15 10:19:32.920989 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Jun 15 10:19:32.920989 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:19:32.920989 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jun 15 10:19:32.920989 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jun 15 10:19:32.920989 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Jun 15 10:19:32.920989 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Jun 15 10:19:32.920989 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 15 10:19:32.920989 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application next(self.gen) Jun 15 10:19:32.922010 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jun 15 10:19:32.922010 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application yield resource Jun 15 10:19:32.922010 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 15 10:19:32.922010 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application next(self.gen) Jun 15 10:19:32.922010 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jun 15 10:19:32.922010 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application self.session.rollback() Jun 15 10:19:32.922010 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jun 15 10:19:32.922010 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application self.force_reraise() Jun 15 10:19:32.922010 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jun 15 10:19:32.922010 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application raise self.value Jun 15 10:19:32.922010 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jun 15 10:19:32.922010 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jun 15 10:19:32.922010 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jun 15 10:19:32.922010 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application session.commit() Jun 15 10:19:32.922010 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jun 15 10:19:32.922010 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jun 15 10:19:32.922010 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jun 15 10:19:32.922924 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application self._prepare_impl() Jun 15 10:19:32.922924 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jun 15 10:19:32.922924 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application self.session.flush() Jun 15 10:19:32.922924 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jun 15 10:19:32.922924 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application self._flush(objects) Jun 15 10:19:32.922924 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jun 15 10:19:32.922924 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jun 15 10:19:32.922924 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jun 15 10:19:32.922924 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application compat.raise_( Jun 15 10:19:32.922924 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 15 10:19:32.922924 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application raise exception Jun 15 10:19:32.922924 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jun 15 10:19:32.922924 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application flush_context.execute() Jun 15 10:19:32.922924 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jun 15 10:19:32.922924 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application rec.execute(self) Jun 15 10:19:32.922924 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jun 15 10:19:32.922924 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jun 15 10:19:32.923889 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jun 15 10:19:32.923889 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application _emit_insert_statements( Jun 15 10:19:32.923889 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Jun 15 10:19:32.923889 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application result = connection._execute_20( Jun 15 10:19:32.923889 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jun 15 10:19:32.923889 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jun 15 10:19:32.923889 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection Jun 15 10:19:32.923889 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jun 15 10:19:32.923889 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jun 15 10:19:32.923889 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application ret = self._execute_context( Jun 15 10:19:32.923889 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jun 15 10:19:32.923889 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jun 15 10:19:32.923889 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jun 15 10:19:32.923889 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jun 15 10:19:32.923889 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 15 10:19:32.923889 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application raise exception Jun 15 10:19:32.925019 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 15 10:19:32.925019 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 15 10:19:32.925019 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 15 10:19:32.925019 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 15 10:19:32.925019 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 15 10:19:32.925019 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application result = self._query(query) Jun 15 10:19:32.925019 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 15 10:19:32.925019 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application conn.query(q) Jun 15 10:19:32.925019 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 15 10:19:32.925019 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 15 10:19:32.925019 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 15 10:19:32.925019 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application result.read() Jun 15 10:19:32.925019 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 15 10:19:32.925019 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 15 10:19:32.925019 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 15 10:19:32.925019 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 15 10:19:32.925868 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 15 10:19:32.925868 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 15 10:19:32.925868 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 15 10:19:32.925868 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 15 10:19:32.925868 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jun 15 10:19:32.925868 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jun 15 10:19:32.925868 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application [parameters: {'user_id': '72f93a178f4642fcb940c3e2614bbd70', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jun 15 10:19:32.925868 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jun 15 10:19:32.925868 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Jun 15 10:19:32.925868 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 15 10:19:32.925868 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Jun 15 10:19:32.925868 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:19:32.925868 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:19:32.925868 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:19:32.925868 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:19:32.926675 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:19:32.926675 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:19:32.926675 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:19:32.926675 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:19:32.926675 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:19:32.926675 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:19:32.926675 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:19:32.926675 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Jun 15 10:19:32.926675 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Jun 15 10:19:32.926675 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:19:32.926675 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:19:32.926675 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 15 10:19:32.926675 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 10:19:32.926675 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 15 10:19:32.926675 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 10:19:32.926675 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Jun 15 10:19:32.926675 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Jun 15 10:19:32.929569 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Jun 15 10:19:32.929569 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Jun 15 10:19:32.929569 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jun 15 10:19:32.929569 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jun 15 10:19:32.929569 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jun 15 10:19:32.929569 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Jun 15 10:19:32.929569 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 482/962] 10.222.0.49 () {60 vars in 1131 bytes} [Wed Jun 15 10:19:32 2022] POST /identity/v3/users => generated 160 bytes in 185 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Jun 15 10:19:32.929569 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503faab1-0045-42a2-b1b8-0532b88c1dac None None] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:32.929569 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503faab1-0045-42a2-b1b8-0532b88c1dac None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:32.929569 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503faab1-0045-42a2-b1b8-0532b88c1dac None None] PATH_INFO: `/` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:32.937988 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 483/963] 10.222.0.49 () {54 vars in 926 bytes} [Wed Jun 15 10:19:32 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:19:32.937988 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503faab1-0045-42a2-b1b8-0532b88c1dac None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:19:32.945402 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:19:32.948076 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575c9e58-647c-424e-ae48-1e9c395361a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:19:32.948991 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575c9e58-647c-424e-ae48-1e9c395361a0 None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:32.949191 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575c9e58-647c-424e-ae48-1e9c395361a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:32.949384 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575c9e58-647c-424e-ae48-1e9c395361a0 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:32.950230 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-575c9e58-647c-424e-ae48-1e9c395361a0 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:19:32.951714 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-575c9e58-647c-424e-ae48-1e9c395361a0 None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:19:32.964809 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application [None req-575c9e58-647c-424e-ae48-1e9c395361a0 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 15 10:19:32.964809 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:19:32.964809 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 15 10:19:32.964809 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 15 10:19:32.964809 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 15 10:19:32.964809 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 15 10:19:32.964809 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 15 10:19:32.964809 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 10:19:32.964809 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 15 10:19:32.964809 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application Jun 15 10:19:32.964809 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 15 10:19:32.964809 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application Jun 15 10:19:32.964809 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:19:32.964809 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:19:32.964809 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:19:32.964809 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:19:32.964809 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:19:32.965937 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:19:32.965937 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:19:32.965937 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:19:32.965937 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:19:32.965937 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:19:32.965937 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:19:32.965937 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 15 10:19:32.965937 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 15 10:19:32.965937 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 15 10:19:32.965937 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 15 10:19:32.965937 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:19:32.965937 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:19:32.965937 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:19:32.965937 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:19:32.965937 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:19:32.965937 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:19:32.966806 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:19:32.966806 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application with Lock( Jun 15 10:19:32.966806 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:19:32.966806 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:19:32.966806 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:19:32.966806 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:19:32.966806 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:19:32.966806 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:19:32.966806 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:19:32.966806 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:19:32.966806 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 15 10:19:32.966806 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 10:19:32.966806 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 15 10:19:32.966806 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application Jun 15 10:19:32.966806 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 481/964] 10.222.0.49 () {56 vars in 1126 bytes} [Wed Jun 15 10:19:32 2022] GET /identity/v3/services/object-store => generated 93 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 10:19:32.967688 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee258674-7342-4aea-a72d-9a5e5e469283 None None] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:32.967688 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee258674-7342-4aea-a72d-9a5e5e469283 None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:32.967688 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee258674-7342-4aea-a72d-9a5e5e469283 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:32.984250 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.auth.core [None req-ee258674-7342-4aea-a72d-9a5e5e469283 None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:19:32.985447 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-ee258674-7342-4aea-a72d-9a5e5e469283 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:19:32.997185 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=100 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:19:33.055321 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503faab1-0045-42a2-b1b8-0532b88c1dac None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:19:33.062355 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 482/965] 10.222.0.49 () {58 vars in 1023 bytes} [Wed Jun 15 10:19:32 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 10:19:33.062355 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee258674-7342-4aea-a72d-9a5e5e469283 None None] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:19:33.093706 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3623e7-e4af-4ebb-bc44-0a4e29c418c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:19:33.094507 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3623e7-e4af-4ebb-bc44-0a4e29c418c1 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:33.094798 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3623e7-e4af-4ebb-bc44-0a4e29c418c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:33.095077 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3623e7-e4af-4ebb-bc44-0a4e29c418c1 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:33.103862 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3623e7-e4af-4ebb-bc44-0a4e29c418c1 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:19:33.105548 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3623e7-e4af-4ebb-bc44-0a4e29c418c1 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:19:33.121228 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application [None req-bf3623e7-e4af-4ebb-bc44-0a4e29c418c1 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 15 10:19:33.121228 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:19:33.121228 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:19:33.121228 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:19:33.121228 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:19:33.121228 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:19:33.121228 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:19:33.121228 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:19:33.121228 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:19:33.121228 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:19:33.121228 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:19:33.121228 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:19:33.121228 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 15 10:19:33.121228 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 15 10:19:33.121228 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 15 10:19:33.121228 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 15 10:19:33.122367 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:19:33.122367 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:19:33.122367 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 15 10:19:33.122367 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 10:19:33.122367 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 15 10:19:33.122367 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 10:19:33.122367 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:19:33.122367 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:19:33.122367 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:19:33.122367 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:19:33.122367 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:19:33.122367 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application with Lock( Jun 15 10:19:33.122367 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:19:33.122367 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:19:33.122367 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:19:33.122367 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:19:33.122367 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:19:33.122367 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:19:33.123847 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:19:33.123847 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:19:33.123847 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 15 10:19:33.123847 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 15 10:19:33.123847 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 15 10:19:33.123847 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 15 10:19:33.123847 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 15 10:19:33.123847 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 15 10:19:33.123847 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 15 10:19:33.123847 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Jun 15 10:19:33.123847 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 484/966] 10.222.0.49 () {56 vars in 1099 bytes} [Wed Jun 15 10:19:32 2022] GET /identity/v3/users/cinder => generated 84 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:19:33.123847 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3623e7-e4af-4ebb-bc44-0a4e29c418c1 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:19:33.169562 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee258674-7342-4aea-a72d-9a5e5e469283 None None] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:19:33.189643 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:19:33.206934 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48a6384-b07d-4e5f-afa1-fe1fdfdb0b1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:19:33.207806 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48a6384-b07d-4e5f-afa1-fe1fdfdb0b1f None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:33.208090 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48a6384-b07d-4e5f-afa1-fe1fdfdb0b1f None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:33.208732 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48a6384-b07d-4e5f-afa1-fe1fdfdb0b1f None admin] PATH_INFO: `/v3/services` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:33.209711 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48a6384-b07d-4e5f-afa1-fe1fdfdb0b1f None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:19:33.210021 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48a6384-b07d-4e5f-afa1-fe1fdfdb0b1f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:19:33.211894 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48a6384-b07d-4e5f-afa1-fe1fdfdb0b1f None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:19:33.222651 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 483/967] 10.222.0.49 () {56 vars in 1122 bytes} [Wed Jun 15 10:19:33 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:19:33.222651 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48a6384-b07d-4e5f-afa1-fe1fdfdb0b1f None admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:19:33.225627 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3623e7-e4af-4ebb-bc44-0a4e29c418c1 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:19:33.236232 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=100 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:19:33.262083 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a18ff85-8202-4d46-b1f4-78b02ab05495 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:19:33.262897 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a18ff85-8202-4d46-b1f4-78b02ab05495 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:33.263208 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a18ff85-8202-4d46-b1f4-78b02ab05495 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:33.263489 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a18ff85-8202-4d46-b1f4-78b02ab05495 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:33.273887 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a18ff85-8202-4d46-b1f4-78b02ab05495 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 10:19:33.282316 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application [None req-8a18ff85-8202-4d46-b1f4-78b02ab05495 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 15 10:19:33.282316 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:19:33.282316 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:19:33.282316 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:19:33.282316 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:19:33.282316 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:19:33.282316 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:19:33.282316 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:19:33.282316 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:19:33.282316 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:19:33.282316 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:19:33.282316 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:19:33.282316 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 10:19:33.282316 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 10:19:33.282316 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 10:19:33.282316 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 10:19:33.283487 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 10:19:33.283487 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 10:19:33.283487 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 10:19:33.283487 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 10:19:33.283487 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:19:33.283487 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:19:33.283487 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:19:33.283487 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:19:33.283487 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:19:33.283487 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:19:33.283487 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:19:33.283487 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application with Lock( Jun 15 10:19:33.283487 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:19:33.283487 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:19:33.283487 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:19:33.283487 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:19:33.284358 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:19:33.284358 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:19:33.284358 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:19:33.284358 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:19:33.284358 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 10:19:33.284358 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 10:19:33.284358 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 10:19:33.284358 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 10:19:33.284358 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 10:19:33.284358 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 10:19:33.284358 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 15 10:19:33.284358 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Jun 15 10:19:33.284358 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 485/968] 10.222.0.49 () {56 vars in 1120 bytes} [Wed Jun 15 10:19:33 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:19:33.284358 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a18ff85-8202-4d46-b1f4-78b02ab05495 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:19:33.337755 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48a6384-b07d-4e5f-afa1-fe1fdfdb0b1f None admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:19:33.373231 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3116d0-59b0-4871-ab4f-f619bc3e5c11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:19:33.373827 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3116d0-59b0-4871-ab4f-f619bc3e5c11 None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:33.374113 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3116d0-59b0-4871-ab4f-f619bc3e5c11 None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:33.374390 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3116d0-59b0-4871-ab4f-f619bc3e5c11 None admin] PATH_INFO: `/v3/users` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:33.375216 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc3116d0-59b0-4871-ab4f-f619bc3e5c11 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=80119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:19:33.375526 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc3116d0-59b0-4871-ab4f-f619bc3e5c11 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:19:33.376958 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc3116d0-59b0-4871-ab4f-f619bc3e5c11 None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:19:33.393657 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a18ff85-8202-4d46-b1f4-78b02ab05495 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:19:33.411933 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 484/969] 10.222.0.49 () {56 vars in 1101 bytes} [Wed Jun 15 10:19:33 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:19:33.422490 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:19:33.424110 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3116d0-59b0-4871-ab4f-f619bc3e5c11 None admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:19:33.447463 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44688f58-39ca-43e5-9d9a-da5efb4cea78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:19:33.448219 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44688f58-39ca-43e5-9d9a-da5efb4cea78 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:33.448504 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44688f58-39ca-43e5-9d9a-da5efb4cea78 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:33.448775 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44688f58-39ca-43e5-9d9a-da5efb4cea78 None admin] PATH_INFO: `/v3/services` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:33.449568 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44688f58-39ca-43e5-9d9a-da5efb4cea78 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:19:33.450542 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44688f58-39ca-43e5-9d9a-da5efb4cea78 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:19:33.452346 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44688f58-39ca-43e5-9d9a-da5efb4cea78 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:19:33.461585 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 486/970] 10.222.0.49 () {56 vars in 1122 bytes} [Wed Jun 15 10:19:33 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:19:33.467223 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44688f58-39ca-43e5-9d9a-da5efb4cea78 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:19:33.503969 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=100 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:19:33.554190 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3116d0-59b0-4871-ab4f-f619bc3e5c11 None admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:19:33.602687 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44688f58-39ca-43e5-9d9a-da5efb4cea78 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:19:33.613271 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317ee30b-626e-4cf6-9080-b15321e577c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:19:33.614897 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317ee30b-626e-4cf6-9080-b15321e577c2 None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:33.615393 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317ee30b-626e-4cf6-9080-b15321e577c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:33.615869 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317ee30b-626e-4cf6-9080-b15321e577c2 None admin] PATH_INFO: `/v3/roles` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:33.617430 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-317ee30b-626e-4cf6-9080-b15321e577c2 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:19:33.617929 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-317ee30b-626e-4cf6-9080-b15321e577c2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:19:33.620102 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-317ee30b-626e-4cf6-9080-b15321e577c2 None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:19:33.630833 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 485/971] 10.222.0.49 () {56 vars in 1115 bytes} [Wed Jun 15 10:19:33 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:19:33.636544 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317ee30b-626e-4cf6-9080-b15321e577c2 None admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:19:33.645994 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1df488-fa84-4e25-a5f4-c518e4479939 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:19:33.646668 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1df488-fa84-4e25-a5f4-c518e4479939 None admin] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:33.647283 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1df488-fa84-4e25-a5f4-c518e4479939 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:33.647283 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1df488-fa84-4e25-a5f4-c518e4479939 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:33.648083 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d1df488-fa84-4e25-a5f4-c518e4479939 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:19:33.649528 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d1df488-fa84-4e25-a5f4-c518e4479939 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:19:33.672163 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 487/972] 10.222.0.49 () {60 vars in 1144 bytes} [Wed Jun 15 10:19:33 2022] POST /identity/v3/endpoints => generated 349 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 10:19:33.761755 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317ee30b-626e-4cf6-9080-b15321e577c2 None admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:19:33.798269 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=100 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:19:33.812062 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560dd9af-7369-4751-987c-05a5665cc312 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:19:33.812929 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560dd9af-7369-4751-987c-05a5665cc312 None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:33.813217 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560dd9af-7369-4751-987c-05a5665cc312 None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:33.813493 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560dd9af-7369-4751-987c-05a5665cc312 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:33.825599 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-560dd9af-7369-4751-987c-05a5665cc312 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:19:33.827112 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-560dd9af-7369-4751-987c-05a5665cc312 None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:19:33.846468 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application [None req-560dd9af-7369-4751-987c-05a5665cc312 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 10:19:33.846468 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:19:33.846468 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:19:33.846468 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:19:33.846468 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:19:33.846468 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:19:33.846468 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:19:33.846468 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:19:33.846468 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:19:33.846468 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:19:33.846468 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:19:33.846468 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:19:33.846468 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 10:19:33.846468 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 10:19:33.846468 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 10:19:33.847656 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 10:19:33.847656 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:19:33.847656 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:19:33.847656 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:19:33.847656 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:19:33.847656 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:19:33.847656 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:19:33.847656 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:19:33.847656 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application with Lock( Jun 15 10:19:33.847656 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:19:33.847656 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:19:33.847656 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:19:33.847656 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:19:33.847656 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:19:33.847656 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:19:33.847656 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:19:33.847656 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:19:33.848760 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 10:19:33.848760 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 10:19:33.848760 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 10:19:33.848760 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application Jun 15 10:19:33.848760 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 486/973] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Jun 15 10:19:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 10:19:33.854729 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1df488-fa84-4e25-a5f4-c518e4479939 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:19:33.908526 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:19:33.915010 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4baea4-7c04-4e28-bb20-e088e3e2e61c None None] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:33.915367 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4baea4-7c04-4e28-bb20-e088e3e2e61c None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:33.915968 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4baea4-7c04-4e28-bb20-e088e3e2e61c None None] PATH_INFO: `/` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:33.917000 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 487/974] 10.222.0.49 () {54 vars in 926 bytes} [Wed Jun 15 10:19:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 15 10:19:33.922152 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9072a969-bded-4388-bec2-6d6cb4d455bf None None] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:33.922443 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9072a969-bded-4388-bec2-6d6cb4d455bf None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:33.922736 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9072a969-bded-4388-bec2-6d6cb4d455bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:33.940311 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.auth.core [None req-9072a969-bded-4388-bec2-6d6cb4d455bf None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:19:33.941497 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-9072a969-bded-4388-bec2-6d6cb4d455bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:19:33.967663 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1df488-fa84-4e25-a5f4-c518e4479939 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:19:34.001872 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb196ee-8014-4a0c-81a2-17470d7da7df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:19:34.002543 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb196ee-8014-4a0c-81a2-17470d7da7df None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:34.002832 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb196ee-8014-4a0c-81a2-17470d7da7df None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:34.003137 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb196ee-8014-4a0c-81a2-17470d7da7df None admin] PATH_INFO: `/v3/domains` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:34.003865 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb196ee-8014-4a0c-81a2-17470d7da7df None admin] RBAC: Adding query filter params (name=Default) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:19:34.004172 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb196ee-8014-4a0c-81a2-17470d7da7df None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:19:34.005539 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb196ee-8014-4a0c-81a2-17470d7da7df None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:19:34.014300 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 488/975] 10.222.0.49 () {58 vars in 1023 bytes} [Wed Jun 15 10:19:33 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 15 10:19:34.022463 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 488/976] 10.222.0.49 () {56 vars in 1109 bytes} [Wed Jun 15 10:19:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:19:34.024051 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9072a969-bded-4388-bec2-6d6cb4d455bf None None] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:19:34.038983 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb196ee-8014-4a0c-81a2-17470d7da7df None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:19:34.129233 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9072a969-bded-4388-bec2-6d6cb4d455bf None None] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:19:34.139650 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=100 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:19:34.150769 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb196ee-8014-4a0c-81a2-17470d7da7df None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:19:34.159066 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:19:34.168072 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744e7dfd-7899-4d18-b8f0-a1c7df6af674 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:19:34.168897 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744e7dfd-7899-4d18-b8f0-a1c7df6af674 None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:34.169186 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744e7dfd-7899-4d18-b8f0-a1c7df6af674 None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:34.169463 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744e7dfd-7899-4d18-b8f0-a1c7df6af674 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:34.179244 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-744e7dfd-7899-4d18-b8f0-a1c7df6af674 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:19:34.180712 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-744e7dfd-7899-4d18-b8f0-a1c7df6af674 None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:19:34.187440 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee6abda-cebc-4623-8255-1c47689271c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:19:34.188193 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee6abda-cebc-4623-8255-1c47689271c7 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:34.188475 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee6abda-cebc-4623-8255-1c47689271c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:34.188749 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee6abda-cebc-4623-8255-1c47689271c7 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:34.189393 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee6abda-cebc-4623-8255-1c47689271c7 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:19:34.190747 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee6abda-cebc-4623-8255-1c47689271c7 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:19:34.195974 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application [None req-744e7dfd-7899-4d18-b8f0-a1c7df6af674 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 15 10:19:34.195974 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:19:34.195974 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:19:34.195974 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:19:34.195974 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:19:34.195974 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:19:34.195974 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:19:34.195974 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:19:34.195974 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:19:34.195974 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:19:34.195974 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:19:34.195974 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:19:34.195974 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 15 10:19:34.195974 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 15 10:19:34.195974 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 15 10:19:34.195974 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 15 10:19:34.197511 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:19:34.197511 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:19:34.197511 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 15 10:19:34.197511 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 10:19:34.197511 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 15 10:19:34.197511 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 10:19:34.197511 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:19:34.197511 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:19:34.197511 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:19:34.197511 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:19:34.197511 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:19:34.197511 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application with Lock( Jun 15 10:19:34.197511 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:19:34.197511 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:19:34.197511 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:19:34.197511 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:19:34.197511 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:19:34.197511 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:19:34.198501 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:19:34.198501 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:19:34.198501 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 15 10:19:34.198501 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 15 10:19:34.198501 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 15 10:19:34.198501 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 15 10:19:34.198501 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 15 10:19:34.198501 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 15 10:19:34.198501 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 15 10:19:34.198501 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application Jun 15 10:19:34.202528 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 489/977] 10.222.0.49 () {56 vars in 1117 bytes} [Wed Jun 15 10:19:34 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:19:34.202986 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application [None req-8ee6abda-cebc-4623-8255-1c47689271c7 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jun 15 10:19:34.202986 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:19:34.202986 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 15 10:19:34.202986 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 15 10:19:34.202986 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 15 10:19:34.202986 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 15 10:19:34.202986 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 15 10:19:34.202986 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 10:19:34.202986 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 15 10:19:34.202986 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Jun 15 10:19:34.202986 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 15 10:19:34.202986 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Jun 15 10:19:34.202986 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:19:34.202986 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:19:34.202986 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:19:34.202986 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:19:34.202986 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:19:34.204648 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:19:34.204648 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:19:34.204648 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:19:34.204648 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:19:34.204648 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:19:34.204648 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:19:34.204648 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 15 10:19:34.204648 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 15 10:19:34.204648 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 15 10:19:34.204648 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 15 10:19:34.204648 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:19:34.204648 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:19:34.204648 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:19:34.204648 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:19:34.204648 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:19:34.204648 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:19:34.206583 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:19:34.206583 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application with Lock( Jun 15 10:19:34.206583 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:19:34.206583 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:19:34.206583 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:19:34.206583 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:19:34.206583 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:19:34.206583 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:19:34.206583 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:19:34.206583 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:19:34.206583 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 15 10:19:34.206583 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 10:19:34.206583 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 15 10:19:34.206583 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Jun 15 10:19:34.208364 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744e7dfd-7899-4d18-b8f0-a1c7df6af674 None admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:19:34.208515 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 489/978] 10.222.0.49 () {56 vars in 1111 bytes} [Wed Jun 15 10:19:34 2022] GET /identity/v3/services/compute => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 10:19:34.213422 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee6abda-cebc-4623-8255-1c47689271c7 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:19:34.322636 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee6abda-cebc-4623-8255-1c47689271c7 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:19:34.324123 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744e7dfd-7899-4d18-b8f0-a1c7df6af674 None admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:19:34.357211 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11f0e43-7c58-4fc7-ae71-07caa4ba853f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:19:34.357875 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11f0e43-7c58-4fc7-ae71-07caa4ba853f None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:34.358158 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11f0e43-7c58-4fc7-ae71-07caa4ba853f None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:34.358429 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11f0e43-7c58-4fc7-ae71-07caa4ba853f None admin] PATH_INFO: `/v3/services` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:34.359207 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e11f0e43-7c58-4fc7-ae71-07caa4ba853f None admin] RBAC: Adding query filter params (name=compute) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:19:34.359514 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e11f0e43-7c58-4fc7-ae71-07caa4ba853f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:19:34.360872 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e11f0e43-7c58-4fc7-ae71-07caa4ba853f None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:19:34.362004 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81637932-8f04-4e86-aa1d-5d120078f79b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:19:34.362983 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81637932-8f04-4e86-aa1d-5d120078f79b None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:34.363328 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81637932-8f04-4e86-aa1d-5d120078f79b None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:34.363616 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81637932-8f04-4e86-aa1d-5d120078f79b None admin] PATH_INFO: `/v3/users` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:34.364454 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81637932-8f04-4e86-aa1d-5d120078f79b None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:19:34.364766 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81637932-8f04-4e86-aa1d-5d120078f79b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:19:34.370829 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 490/979] 10.222.0.49 () {56 vars in 1112 bytes} [Wed Jun 15 10:19:34 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 10:19:34.370967 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81637932-8f04-4e86-aa1d-5d120078f79b None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:19:34.373276 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11f0e43-7c58-4fc7-ae71-07caa4ba853f None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:19:34.387423 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 490/980] 10.222.0.49 () {56 vars in 1149 bytes} [Wed Jun 15 10:19:34 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:19:34.392864 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81637932-8f04-4e86-aa1d-5d120078f79b None admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:19:34.411793 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:19:34.419806 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=100 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:19:34.484397 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11f0e43-7c58-4fc7-ae71-07caa4ba853f None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:19:34.498295 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81637932-8f04-4e86-aa1d-5d120078f79b None admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:19:34.519535 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba90a5a-3842-4a1d-9164-7e1fc00827c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:19:34.520313 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba90a5a-3842-4a1d-9164-7e1fc00827c4 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:34.520701 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba90a5a-3842-4a1d-9164-7e1fc00827c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:34.521015 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba90a5a-3842-4a1d-9164-7e1fc00827c4 None admin] PATH_INFO: `/v3/services` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:34.521847 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba90a5a-3842-4a1d-9164-7e1fc00827c4 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:19:34.522194 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba90a5a-3842-4a1d-9164-7e1fc00827c4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:19:34.523751 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba90a5a-3842-4a1d-9164-7e1fc00827c4 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:19:34.528642 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 491/981] 10.222.0.49 () {56 vars in 1112 bytes} [Wed Jun 15 10:19:34 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 10:19:34.534281 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba90a5a-3842-4a1d-9164-7e1fc00827c4 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:19:34.540794 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e31b62-e338-463c-a18c-b104d260e2a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:19:34.541656 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e31b62-e338-463c-a18c-b104d260e2a6 None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:34.542100 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e31b62-e338-463c-a18c-b104d260e2a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:34.542528 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e31b62-e338-463c-a18c-b104d260e2a6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:34.615201 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e31b62-e338-463c-a18c-b104d260e2a6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:19:34.616893 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e31b62-e338-463c-a18c-b104d260e2a6 None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:19:34.628594 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:19:34.633073 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba90a5a-3842-4a1d-9164-7e1fc00827c4 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:19:34.645938 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=100 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:19:34.669846 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afcc5221-366c-441d-829d-94ec27db1b92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:19:34.670436 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcc5221-366c-441d-829d-94ec27db1b92 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:34.670729 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcc5221-366c-441d-829d-94ec27db1b92 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:34.671007 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcc5221-366c-441d-829d-94ec27db1b92 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:34.671856 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afcc5221-366c-441d-829d-94ec27db1b92 None admin] RBAC: Adding query filter params (interface=public, service_id=a5d8ec83a9994f808ed8fca22368c884, region_id=RegionOne) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:19:34.672172 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afcc5221-366c-441d-829d-94ec27db1b92 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:19:34.673534 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afcc5221-366c-441d-829d-94ec27db1b92 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:19:34.678681 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 492/982] 10.222.0.49 () {56 vars in 1251 bytes} [Wed Jun 15 10:19:34 2022] GET /identity/v3/endpoints?service_id=a5d8ec83a9994f808ed8fca22368c884&interface=public®ion_id=RegionOne => generated 194 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 10:19:34.684773 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afcc5221-366c-441d-829d-94ec27db1b92 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:19:34.737430 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 491/983] 10.222.0.49 () {58 vars in 1282 bytes} [Wed Jun 15 10:19:34 2022] GET /identity/v3/auth/tokens => generated 1560 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 15 10:19:34.742881 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e31b62-e338-463c-a18c-b104d260e2a6 None admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:19:34.803854 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afcc5221-366c-441d-829d-94ec27db1b92 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:19:34.839981 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9bb2138-c1d3-46c9-bd74-394a49ed2b22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:19:34.840800 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bb2138-c1d3-46c9-bd74-394a49ed2b22 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:34.841107 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bb2138-c1d3-46c9-bd74-394a49ed2b22 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:34.841394 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bb2138-c1d3-46c9-bd74-394a49ed2b22 None admin] PATH_INFO: `/v3/services` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:34.842377 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9bb2138-c1d3-46c9-bd74-394a49ed2b22 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:19:34.843774 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9bb2138-c1d3-46c9-bd74-394a49ed2b22 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:19:34.863120 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 493/984] 10.222.0.49 () {56 vars in 1087 bytes} [Wed Jun 15 10:19:34 2022] GET /identity/v3/services => generated 1727 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 10:19:34.863335 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e31b62-e338-463c-a18c-b104d260e2a6 None admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:19:34.888375 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=100 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:19:34.898745 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24fe3df-e836-41b3-8ddb-6f4d96b34354 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:19:34.899751 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24fe3df-e836-41b3-8ddb-6f4d96b34354 None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:34.900106 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24fe3df-e836-41b3-8ddb-6f4d96b34354 None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:34.900458 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24fe3df-e836-41b3-8ddb-6f4d96b34354 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:34.912474 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24fe3df-e836-41b3-8ddb-6f4d96b34354 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:19:34.914010 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24fe3df-e836-41b3-8ddb-6f4d96b34354 None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:19:34.934843 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application [None req-c24fe3df-e836-41b3-8ddb-6f4d96b34354 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 10:19:34.934843 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:19:34.934843 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:19:34.934843 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:19:34.934843 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:19:34.934843 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:19:34.934843 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:19:34.934843 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:19:34.934843 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:19:34.934843 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:19:34.934843 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:19:34.934843 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:19:34.934843 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 10:19:34.934843 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 10:19:34.934843 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 10:19:34.935990 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 10:19:34.935990 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:19:34.935990 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:19:34.935990 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:19:34.935990 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:19:34.935990 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:19:34.935990 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:19:34.935990 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:19:34.935990 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application with Lock( Jun 15 10:19:34.935990 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:19:34.935990 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:19:34.935990 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:19:34.935990 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:19:34.935990 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:19:34.935990 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:19:34.935990 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:19:34.935990 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:19:34.937222 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 10:19:34.937222 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 10:19:34.937222 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 10:19:34.937222 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application Jun 15 10:19:34.937222 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 492/985] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Jun 15 10:19:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:19:34.941765 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9bb2138-c1d3-46c9-bd74-394a49ed2b22 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:19:34.971486 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:19:35.051597 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9bb2138-c1d3-46c9-bd74-394a49ed2b22 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:19:35.084738 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a493fab6-b1a6-4faf-b4a7-53664fd6469f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:19:35.085394 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a493fab6-b1a6-4faf-b4a7-53664fd6469f None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:35.085680 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a493fab6-b1a6-4faf-b4a7-53664fd6469f None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:35.085950 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a493fab6-b1a6-4faf-b4a7-53664fd6469f None admin] PATH_INFO: `/v3/domains` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:35.086717 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a493fab6-b1a6-4faf-b4a7-53664fd6469f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:19:35.087021 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a493fab6-b1a6-4faf-b4a7-53664fd6469f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:19:35.088951 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a493fab6-b1a6-4faf-b4a7-53664fd6469f None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:19:35.101263 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 494/986] 10.222.0.49 () {56 vars in 1109 bytes} [Wed Jun 15 10:19:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:19:35.107172 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24fe3df-e836-41b3-8ddb-6f4d96b34354 None admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:19:35.205755 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24fe3df-e836-41b3-8ddb-6f4d96b34354 None admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:19:35.240170 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f87db9-532a-42d4-8ebf-6ab136d91b46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:19:35.240959 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f87db9-532a-42d4-8ebf-6ab136d91b46 None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:35.241240 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f87db9-532a-42d4-8ebf-6ab136d91b46 None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:35.241509 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f87db9-532a-42d4-8ebf-6ab136d91b46 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:35.248094 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f87db9-532a-42d4-8ebf-6ab136d91b46 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:19:35.249569 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f87db9-532a-42d4-8ebf-6ab136d91b46 None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:19:35.262974 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application [None req-a1f87db9-532a-42d4-8ebf-6ab136d91b46 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 10:19:35.262974 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:19:35.262974 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:19:35.262974 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:19:35.262974 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:19:35.262974 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:19:35.262974 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:19:35.262974 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:19:35.262974 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:19:35.262974 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:19:35.262974 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:19:35.262974 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:19:35.262974 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 15 10:19:35.262974 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 15 10:19:35.262974 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 15 10:19:35.263909 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 15 10:19:35.263909 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:19:35.263909 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:19:35.263909 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:19:35.263909 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:19:35.263909 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:19:35.263909 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:19:35.263909 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:19:35.263909 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application with Lock( Jun 15 10:19:35.263909 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:19:35.263909 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:19:35.263909 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:19:35.263909 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:19:35.263909 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:19:35.263909 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:19:35.263909 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:19:35.263909 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:19:35.264953 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 15 10:19:35.264953 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 15 10:19:35.264953 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 15 10:19:35.264953 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 15 10:19:35.264953 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 15 10:19:35.264953 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 15 10:19:35.264953 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 10:19:35.264953 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application Jun 15 10:19:35.267158 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 493/987] 10.222.0.49 () {56 vars in 1111 bytes} [Wed Jun 15 10:19:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:19:35.269867 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a493fab6-b1a6-4faf-b4a7-53664fd6469f None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:19:35.359709 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:19:35.376330 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a493fab6-b1a6-4faf-b4a7-53664fd6469f None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:19:35.411214 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9729c6-d290-4fef-a247-07a19506a3a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:19:35.411882 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9729c6-d290-4fef-a247-07a19506a3a1 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:35.412168 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9729c6-d290-4fef-a247-07a19506a3a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:35.412444 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9729c6-d290-4fef-a247-07a19506a3a1 None admin] PATH_INFO: `/v3/projects` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:35.413234 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e9729c6-d290-4fef-a247-07a19506a3a1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:19:35.413545 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e9729c6-d290-4fef-a247-07a19506a3a1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:19:35.414972 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e9729c6-d290-4fef-a247-07a19506a3a1 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:19:35.431349 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 495/988] 10.222.0.49 () {56 vars in 1148 bytes} [Wed Jun 15 10:19:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:19:35.432573 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f87db9-532a-42d4-8ebf-6ab136d91b46 None admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:19:35.474504 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=100 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:19:35.536516 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f87db9-532a-42d4-8ebf-6ab136d91b46 None admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:19:35.570482 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9c6fa1-622e-4a73-8e4c-4cc31320718a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:19:35.571337 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9c6fa1-622e-4a73-8e4c-4cc31320718a None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:35.572230 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9c6fa1-622e-4a73-8e4c-4cc31320718a None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:35.572610 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9c6fa1-622e-4a73-8e4c-4cc31320718a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:35.573535 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9c6fa1-622e-4a73-8e4c-4cc31320718a None admin] RBAC: Adding query filter params (role.id=d962cdb5fdb24466a560f5d7bcb1e034, scope.project.id=559b3e713f5844b6ae37151d56dd419f, user.id=103e10244eca444b86b9c51ff314d445) {{(pid=80119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:19:35.573950 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9c6fa1-622e-4a73-8e4c-4cc31320718a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:19:35.575508 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9c6fa1-622e-4a73-8e4c-4cc31320718a None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:19:35.587182 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 494/989] 10.222.0.49 () {56 vars in 1374 bytes} [Wed Jun 15 10:19:35 2022] GET /identity/v3/role_assignments?user.id=103e10244eca444b86b9c51ff314d445&scope.project.id=559b3e713f5844b6ae37151d56dd419f&role.id=d962cdb5fdb24466a560f5d7bcb1e034 => generated 608 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 10:19:35.831116 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6f5e20-29c8-47ef-9c49-e7cfb39220ff None None] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:35.831638 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6f5e20-29c8-47ef-9c49-e7cfb39220ff None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:35.831718 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6f5e20-29c8-47ef-9c49-e7cfb39220ff None None] PATH_INFO: `/` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:35.833162 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 496/990] 10.222.0.49 () {54 vars in 926 bytes} [Wed Jun 15 10:19:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:19:35.838100 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2a0c6a-abb9-4991-9015-3e63c3028bb0 None None] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:35.838491 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2a0c6a-abb9-4991-9015-3e63c3028bb0 None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:35.838650 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2a0c6a-abb9-4991-9015-3e63c3028bb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:35.855158 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.auth.core [None req-4d2a0c6a-abb9-4991-9015-3e63c3028bb0 None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:19:35.856358 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-4d2a0c6a-abb9-4991-9015-3e63c3028bb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:19:35.922508 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 495/991] 10.222.0.49 () {58 vars in 1023 bytes} [Wed Jun 15 10:19:35 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 10:19:35.945528 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6f5e20-29c8-47ef-9c49-e7cfb39220ff None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:19:36.047251 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6f5e20-29c8-47ef-9c49-e7cfb39220ff None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:19:36.081682 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff942546-14fa-41bc-a0f8-1130becb2ca8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:19:36.082493 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff942546-14fa-41bc-a0f8-1130becb2ca8 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:36.082772 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff942546-14fa-41bc-a0f8-1130becb2ca8 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:36.083043 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff942546-14fa-41bc-a0f8-1130becb2ca8 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:36.083722 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff942546-14fa-41bc-a0f8-1130becb2ca8 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:19:36.085086 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff942546-14fa-41bc-a0f8-1130becb2ca8 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:19:36.097330 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application [None req-ff942546-14fa-41bc-a0f8-1130becb2ca8 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 15 10:19:36.097330 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:19:36.097330 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 15 10:19:36.097330 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 15 10:19:36.097330 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 15 10:19:36.097330 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 15 10:19:36.097330 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 15 10:19:36.097330 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 10:19:36.097330 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 15 10:19:36.097330 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Jun 15 10:19:36.097330 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 15 10:19:36.097330 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Jun 15 10:19:36.097330 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:19:36.097330 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:19:36.097330 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:19:36.097330 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:19:36.097330 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:19:36.098318 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:19:36.098318 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:19:36.098318 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:19:36.098318 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:19:36.098318 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:19:36.098318 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:19:36.098318 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 15 10:19:36.098318 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 15 10:19:36.098318 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 15 10:19:36.098318 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 15 10:19:36.098318 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:19:36.098318 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:19:36.098318 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:19:36.098318 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:19:36.098318 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:19:36.098318 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:19:36.099251 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:19:36.099251 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application with Lock( Jun 15 10:19:36.099251 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:19:36.099251 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:19:36.099251 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:19:36.099251 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:19:36.099251 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:19:36.099251 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:19:36.099251 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:19:36.099251 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:19:36.099251 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 15 10:19:36.099251 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 10:19:36.099251 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 15 10:19:36.099251 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Jun 15 10:19:36.106551 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 497/992] 10.222.0.49 () {56 vars in 1126 bytes} [Wed Jun 15 10:19:35 2022] GET /identity/v3/services/object-store => generated 93 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:19:36.106687 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2a0c6a-abb9-4991-9015-3e63c3028bb0 None None] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:19:36.136012 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=100 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:19:36.204516 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2a0c6a-abb9-4991-9015-3e63c3028bb0 None None] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:19:36.237391 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0332ae-81a0-43e0-918b-1a8ef26b8a73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:19:36.238051 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0332ae-81a0-43e0-918b-1a8ef26b8a73 None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:36.238332 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0332ae-81a0-43e0-918b-1a8ef26b8a73 None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:36.238604 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0332ae-81a0-43e0-918b-1a8ef26b8a73 None admin] PATH_INFO: `/v3/services` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:36.239335 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0332ae-81a0-43e0-918b-1a8ef26b8a73 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:19:36.239641 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0332ae-81a0-43e0-918b-1a8ef26b8a73 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:19:36.240996 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0332ae-81a0-43e0-918b-1a8ef26b8a73 None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:19:36.246425 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 496/993] 10.222.0.49 () {56 vars in 1122 bytes} [Wed Jun 15 10:19:36 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:19:36.251808 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff942546-14fa-41bc-a0f8-1130becb2ca8 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:19:36.264584 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:19:36.352637 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff942546-14fa-41bc-a0f8-1130becb2ca8 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:19:36.385539 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2105ed7-bd85-40b5-a3a7-bf4d5f5dc94a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:19:36.386206 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2105ed7-bd85-40b5-a3a7-bf4d5f5dc94a None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:36.386487 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2105ed7-bd85-40b5-a3a7-bf4d5f5dc94a None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:36.386762 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2105ed7-bd85-40b5-a3a7-bf4d5f5dc94a None admin] PATH_INFO: `/v3/services` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:36.387489 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2105ed7-bd85-40b5-a3a7-bf4d5f5dc94a None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:19:36.387793 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2105ed7-bd85-40b5-a3a7-bf4d5f5dc94a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:19:36.389200 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2105ed7-bd85-40b5-a3a7-bf4d5f5dc94a None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:19:36.394291 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 498/994] 10.222.0.49 () {56 vars in 1122 bytes} [Wed Jun 15 10:19:36 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:19:36.400446 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0332ae-81a0-43e0-918b-1a8ef26b8a73 None admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:19:36.491617 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=100 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:19:36.498849 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0332ae-81a0-43e0-918b-1a8ef26b8a73 None admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:19:36.533604 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2931fc-5bbc-476c-82ea-2e2569ea288a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:19:36.534265 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2931fc-5bbc-476c-82ea-2e2569ea288a None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:36.534545 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2931fc-5bbc-476c-82ea-2e2569ea288a None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:36.534822 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2931fc-5bbc-476c-82ea-2e2569ea288a None admin] PATH_INFO: `/v3/endpoints` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:36.535644 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb2931fc-5bbc-476c-82ea-2e2569ea288a None admin] RBAC: Adding query filter params (interface=admin, service_id=6d4c1066aada4da5851a15eb90fa1748, region_id=RegionOne) {{(pid=80119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:19:36.535958 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb2931fc-5bbc-476c-82ea-2e2569ea288a None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:19:36.537324 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb2931fc-5bbc-476c-82ea-2e2569ea288a None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:19:36.542632 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 497/995] 10.222.0.49 () {56 vars in 1249 bytes} [Wed Jun 15 10:19:36 2022] GET /identity/v3/endpoints?service_id=6d4c1066aada4da5851a15eb90fa1748&interface=admin®ion_id=RegionOne => generated 193 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:19:36.547928 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2105ed7-bd85-40b5-a3a7-bf4d5f5dc94a None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:19:36.622754 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:19:36.649324 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2105ed7-bd85-40b5-a3a7-bf4d5f5dc94a None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:19:36.682819 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4903c4-7a42-46e7-ae2f-0fc60ac4296f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:19:36.683547 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4903c4-7a42-46e7-ae2f-0fc60ac4296f None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:36.683834 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4903c4-7a42-46e7-ae2f-0fc60ac4296f None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:36.684110 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4903c4-7a42-46e7-ae2f-0fc60ac4296f None admin] PATH_INFO: `/v3/services` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:36.684806 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd4903c4-7a42-46e7-ae2f-0fc60ac4296f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:19:36.686215 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd4903c4-7a42-46e7-ae2f-0fc60ac4296f None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:19:36.699127 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 499/996] 10.222.0.49 () {56 vars in 1087 bytes} [Wed Jun 15 10:19:36 2022] GET /identity/v3/services => generated 1727 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 10:19:36.998660 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d6bdbb-23f3-4441-a387-77fb7e247aaa None None] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:36.999181 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d6bdbb-23f3-4441-a387-77fb7e247aaa None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:36.999181 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d6bdbb-23f3-4441-a387-77fb7e247aaa None None] PATH_INFO: `/` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:37.000676 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 498/997] 10.222.0.49 () {54 vars in 926 bytes} [Wed Jun 15 10:19:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:19:37.005775 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfae115f-4654-48b6-a163-569b8d6ea415 None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:37.006177 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfae115f-4654-48b6-a163-569b8d6ea415 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:37.006347 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfae115f-4654-48b6-a163-569b8d6ea415 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:37.024021 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-cfae115f-4654-48b6-a163-569b8d6ea415 None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:19:37.025577 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-cfae115f-4654-48b6-a163-569b8d6ea415 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:19:37.094435 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 500/998] 10.222.0.49 () {58 vars in 1023 bytes} [Wed Jun 15 10:19:37 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 10:19:37.117944 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d6bdbb-23f3-4441-a387-77fb7e247aaa None None] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:19:37.224284 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d6bdbb-23f3-4441-a387-77fb7e247aaa None None] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:19:37.259241 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdcdc150-8e23-4f83-996b-02443c969727 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:19:37.259966 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcdc150-8e23-4f83-996b-02443c969727 None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:37.260253 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcdc150-8e23-4f83-996b-02443c969727 None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:37.260539 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcdc150-8e23-4f83-996b-02443c969727 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:37.261191 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdcdc150-8e23-4f83-996b-02443c969727 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:19:37.262562 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdcdc150-8e23-4f83-996b-02443c969727 None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:19:37.274278 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application [None req-cdcdc150-8e23-4f83-996b-02443c969727 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jun 15 10:19:37.274278 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:19:37.274278 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 15 10:19:37.274278 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 15 10:19:37.274278 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 15 10:19:37.274278 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 15 10:19:37.274278 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 15 10:19:37.274278 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 10:19:37.274278 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 15 10:19:37.274278 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application Jun 15 10:19:37.274278 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 15 10:19:37.274278 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application Jun 15 10:19:37.274278 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:19:37.274278 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:19:37.274278 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:19:37.274278 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:19:37.274278 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:19:37.275978 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:19:37.275978 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:19:37.275978 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:19:37.275978 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:19:37.275978 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:19:37.275978 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:19:37.275978 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 15 10:19:37.275978 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 15 10:19:37.275978 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 15 10:19:37.275978 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 15 10:19:37.275978 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:19:37.275978 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:19:37.275978 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:19:37.275978 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:19:37.275978 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:19:37.275978 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:19:37.276865 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:19:37.276865 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application with Lock( Jun 15 10:19:37.276865 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:19:37.276865 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:19:37.276865 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:19:37.276865 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:19:37.276865 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:19:37.276865 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:19:37.276865 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:19:37.276865 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:19:37.276865 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 15 10:19:37.276865 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 10:19:37.276865 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 15 10:19:37.276865 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application Jun 15 10:19:37.279403 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 499/999] 10.222.0.49 () {56 vars in 1111 bytes} [Wed Jun 15 10:19:37 2022] GET /identity/v3/services/compute => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:19:37.282083 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfae115f-4654-48b6-a163-569b8d6ea415 None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:19:37.351444 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:19:37.386060 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfae115f-4654-48b6-a163-569b8d6ea415 None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:19:37.421977 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b896521-609a-4e5c-86be-e227e92e2ace None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:19:37.422652 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b896521-609a-4e5c-86be-e227e92e2ace None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:37.422945 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b896521-609a-4e5c-86be-e227e92e2ace None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:37.423258 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b896521-609a-4e5c-86be-e227e92e2ace None admin] PATH_INFO: `/v3/services` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:37.423967 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b896521-609a-4e5c-86be-e227e92e2ace None admin] RBAC: Adding query filter params (name=compute) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:19:37.424542 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b896521-609a-4e5c-86be-e227e92e2ace None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:19:37.425913 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b896521-609a-4e5c-86be-e227e92e2ace None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:19:37.435431 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 501/1000] 10.222.0.49 () {56 vars in 1112 bytes} [Wed Jun 15 10:19:37 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 10:19:37.436711 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdcdc150-8e23-4f83-996b-02443c969727 None admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:19:37.464415 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=100 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:19:37.543189 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdcdc150-8e23-4f83-996b-02443c969727 None admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:19:37.576575 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6d3fbd-03f2-4ead-b2d7-5ac0c1348f72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:19:37.577247 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6d3fbd-03f2-4ead-b2d7-5ac0c1348f72 None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:37.577532 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6d3fbd-03f2-4ead-b2d7-5ac0c1348f72 None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:37.577809 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6d3fbd-03f2-4ead-b2d7-5ac0c1348f72 None admin] PATH_INFO: `/v3/services` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:37.578506 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf6d3fbd-03f2-4ead-b2d7-5ac0c1348f72 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:19:37.578813 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf6d3fbd-03f2-4ead-b2d7-5ac0c1348f72 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:19:37.580224 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf6d3fbd-03f2-4ead-b2d7-5ac0c1348f72 None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:19:37.585359 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 500/1001] 10.222.0.49 () {56 vars in 1112 bytes} [Wed Jun 15 10:19:37 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:19:37.591292 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b896521-609a-4e5c-86be-e227e92e2ace None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:19:37.694402 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b896521-609a-4e5c-86be-e227e92e2ace None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:19:37.724897 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:19:37.731430 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76432d57-6180-4ce0-b143-764c08343627 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:19:37.732103 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76432d57-6180-4ce0-b143-764c08343627 None admin] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:37.732209 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e66c43-91eb-4961-99f0-356aa127879c None None] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:37.732389 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76432d57-6180-4ce0-b143-764c08343627 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:37.732513 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e66c43-91eb-4961-99f0-356aa127879c None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:37.732681 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76432d57-6180-4ce0-b143-764c08343627 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:37.732788 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e66c43-91eb-4961-99f0-356aa127879c None None] PATH_INFO: `/` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:37.733434 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76432d57-6180-4ce0-b143-764c08343627 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:19:37.734253 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 501/1002] 10.222.0.49 () {54 vars in 926 bytes} [Wed Jun 15 10:19:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:19:37.734782 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76432d57-6180-4ce0-b143-764c08343627 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:19:37.738800 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ffffc0-daa1-4d35-877f-c2fce0163c75 None None] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:37.739106 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ffffc0-daa1-4d35-877f-c2fce0163c75 None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:37.739404 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ffffc0-daa1-4d35-877f-c2fce0163c75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:37.757164 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 502/1003] 10.222.0.49 () {60 vars in 1144 bytes} [Wed Jun 15 10:19:37 2022] POST /identity/v3/endpoints => generated 334 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 10:19:37.769019 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.auth.core [None req-b5ffffc0-daa1-4d35-877f-c2fce0163c75 None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:19:37.770579 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-b5ffffc0-daa1-4d35-877f-c2fce0163c75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:19:37.862374 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 502/1004] 10.222.0.49 () {58 vars in 1023 bytes} [Wed Jun 15 10:19:37 2022] POST /identity/v3/auth/tokens => generated 1716 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 15 10:19:37.884746 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76432d57-6180-4ce0-b143-764c08343627 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:19:37.996778 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76432d57-6180-4ce0-b143-764c08343627 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:19:38.031111 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a64880-2ba7-447e-b544-549c4895fe5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:19:38.031989 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a64880-2ba7-447e-b544-549c4895fe5d None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:38.032279 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a64880-2ba7-447e-b544-549c4895fe5d None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:38.032581 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a64880-2ba7-447e-b544-549c4895fe5d None admin] PATH_INFO: `/v3/services/image` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:38.033268 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a64880-2ba7-447e-b544-549c4895fe5d None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:19:38.034651 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a64880-2ba7-447e-b544-549c4895fe5d None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:19:38.046698 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application [None req-f3a64880-2ba7-447e-b544-549c4895fe5d None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jun 15 10:19:38.046698 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:19:38.046698 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 15 10:19:38.046698 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 15 10:19:38.046698 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 15 10:19:38.046698 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 15 10:19:38.046698 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 15 10:19:38.046698 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 10:19:38.046698 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 15 10:19:38.046698 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Jun 15 10:19:38.046698 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 15 10:19:38.046698 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Jun 15 10:19:38.046698 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:19:38.046698 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:19:38.046698 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:19:38.046698 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:19:38.046698 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:19:38.047876 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:19:38.047876 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:19:38.047876 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:19:38.047876 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:19:38.047876 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:19:38.047876 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:19:38.047876 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 15 10:19:38.047876 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 15 10:19:38.047876 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 15 10:19:38.047876 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 15 10:19:38.047876 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:19:38.047876 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:19:38.047876 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:19:38.047876 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:19:38.047876 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:19:38.047876 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:19:38.049546 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:19:38.049546 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application with Lock( Jun 15 10:19:38.049546 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:19:38.049546 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:19:38.049546 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:19:38.049546 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:19:38.049546 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:19:38.049546 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:19:38.049546 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:19:38.049546 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:19:38.049546 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 15 10:19:38.049546 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 10:19:38.049546 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 15 10:19:38.049546 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Jun 15 10:19:38.049546 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 503/1005] 10.222.0.49 () {56 vars in 1105 bytes} [Wed Jun 15 10:19:37 2022] GET /identity/v3/services/image => generated 86 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:19:38.054344 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ffffc0-daa1-4d35-877f-c2fce0163c75 None None] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:19:38.070905 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=100 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:19:38.157750 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ffffc0-daa1-4d35-877f-c2fce0163c75 None None] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:19:38.191327 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1714b6c-73c7-42b7-a38a-5f6be582e479 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:19:38.192004 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1714b6c-73c7-42b7-a38a-5f6be582e479 None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:38.192289 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1714b6c-73c7-42b7-a38a-5f6be582e479 None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:38.192562 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1714b6c-73c7-42b7-a38a-5f6be582e479 None admin] PATH_INFO: `/v3/services` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:38.193263 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1714b6c-73c7-42b7-a38a-5f6be582e479 None admin] RBAC: Adding query filter params (name=image) {{(pid=80119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:19:38.193567 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1714b6c-73c7-42b7-a38a-5f6be582e479 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:19:38.194927 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1714b6c-73c7-42b7-a38a-5f6be582e479 None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:19:38.200681 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 503/1006] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Jun 15 10:19:38 2022] GET /identity/v3/services?name=image => generated 122 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:19:38.206049 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a64880-2ba7-447e-b544-549c4895fe5d None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:19:38.259896 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:19:38.310030 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a64880-2ba7-447e-b544-549c4895fe5d None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:19:38.344904 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08db870c-a3f7-4f11-95f7-f6dda38b160f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:19:38.344904 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08db870c-a3f7-4f11-95f7-f6dda38b160f None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:38.344904 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08db870c-a3f7-4f11-95f7-f6dda38b160f None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:38.344904 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08db870c-a3f7-4f11-95f7-f6dda38b160f None admin] PATH_INFO: `/v3/services` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:38.345482 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08db870c-a3f7-4f11-95f7-f6dda38b160f None admin] RBAC: Adding query filter params (type=image) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:19:38.345789 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08db870c-a3f7-4f11-95f7-f6dda38b160f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:19:38.347178 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08db870c-a3f7-4f11-95f7-f6dda38b160f None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:19:38.352081 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 504/1007] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Jun 15 10:19:38 2022] GET /identity/v3/services?type=image => generated 122 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:19:38.848163 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900bfd8e-6168-45b5-902e-bb79c6251971 None None] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:38.848509 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900bfd8e-6168-45b5-902e-bb79c6251971 None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:38.848731 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900bfd8e-6168-45b5-902e-bb79c6251971 None None] PATH_INFO: `/` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:38.850229 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 504/1008] 10.222.0.49 () {54 vars in 926 bytes} [Wed Jun 15 10:19:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:19:38.855783 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20639bc-bf7f-4f66-86d2-fe341b60a443 None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:38.856080 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20639bc-bf7f-4f66-86d2-fe341b60a443 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:38.856364 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20639bc-bf7f-4f66-86d2-fe341b60a443 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:38.873041 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-b20639bc-bf7f-4f66-86d2-fe341b60a443 None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:19:38.874237 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-b20639bc-bf7f-4f66-86d2-fe341b60a443 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:19:38.940616 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 505/1009] 10.222.0.49 () {58 vars in 1023 bytes} [Wed Jun 15 10:19:38 2022] POST /identity/v3/auth/tokens => generated 1716 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 10:19:38.963778 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900bfd8e-6168-45b5-902e-bb79c6251971 None None] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:19:39.044615 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=100 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:19:39.068892 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900bfd8e-6168-45b5-902e-bb79c6251971 None None] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:19:39.105304 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db3fc1b-7767-4365-a12e-1e280ed8dcc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:19:39.106118 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db3fc1b-7767-4365-a12e-1e280ed8dcc7 None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:39.106403 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db3fc1b-7767-4365-a12e-1e280ed8dcc7 None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:39.106681 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db3fc1b-7767-4365-a12e-1e280ed8dcc7 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:39.107358 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db3fc1b-7767-4365-a12e-1e280ed8dcc7 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:19:39.108800 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db3fc1b-7767-4365-a12e-1e280ed8dcc7 None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:19:39.120961 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application [None req-2db3fc1b-7767-4365-a12e-1e280ed8dcc7 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 15 10:19:39.120961 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:19:39.120961 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 15 10:19:39.120961 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 15 10:19:39.120961 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 15 10:19:39.120961 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 15 10:19:39.120961 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 15 10:19:39.120961 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 10:19:39.120961 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 15 10:19:39.120961 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application Jun 15 10:19:39.120961 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 15 10:19:39.120961 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application Jun 15 10:19:39.120961 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:19:39.120961 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:19:39.120961 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:19:39.120961 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:19:39.120961 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:19:39.121962 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:19:39.121962 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:19:39.121962 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:19:39.121962 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:19:39.121962 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:19:39.121962 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:19:39.121962 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 15 10:19:39.121962 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 15 10:19:39.121962 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 15 10:19:39.121962 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 15 10:19:39.121962 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:19:39.121962 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:19:39.121962 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:19:39.121962 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:19:39.121962 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:19:39.121962 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:19:39.122807 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:19:39.122807 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application with Lock( Jun 15 10:19:39.122807 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:19:39.122807 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:19:39.122807 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:19:39.122807 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:19:39.122807 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:19:39.122807 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:19:39.122807 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:19:39.122807 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:19:39.122807 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 15 10:19:39.122807 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 10:19:39.122807 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 15 10:19:39.122807 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application Jun 15 10:19:39.127328 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 505/1010] 10.222.0.49 () {56 vars in 1126 bytes} [Wed Jun 15 10:19:38 2022] GET /identity/v3/services/object-store => generated 93 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:19:39.128180 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20639bc-bf7f-4f66-86d2-fe341b60a443 None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:19:39.177448 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:19:39.230722 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20639bc-bf7f-4f66-86d2-fe341b60a443 None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:19:39.263593 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3c06fa-6acc-418b-aaaa-9ec2bcf41e09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:19:39.264258 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3c06fa-6acc-418b-aaaa-9ec2bcf41e09 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:39.264542 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3c06fa-6acc-418b-aaaa-9ec2bcf41e09 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:39.264817 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3c06fa-6acc-418b-aaaa-9ec2bcf41e09 None admin] PATH_INFO: `/v3/services` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:39.265513 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb3c06fa-6acc-418b-aaaa-9ec2bcf41e09 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:19:39.265822 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb3c06fa-6acc-418b-aaaa-9ec2bcf41e09 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:19:39.267212 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb3c06fa-6acc-418b-aaaa-9ec2bcf41e09 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:19:39.272091 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 506/1011] 10.222.0.49 () {56 vars in 1122 bytes} [Wed Jun 15 10:19:39 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 10:19:39.278084 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db3fc1b-7767-4365-a12e-1e280ed8dcc7 None admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:19:39.376521 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db3fc1b-7767-4365-a12e-1e280ed8dcc7 None admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:19:39.410101 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5661bf2-3cf2-48b6-90e7-940dce82a286 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:19:39.410770 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5661bf2-3cf2-48b6-90e7-940dce82a286 None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:39.411055 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5661bf2-3cf2-48b6-90e7-940dce82a286 None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:39.411356 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5661bf2-3cf2-48b6-90e7-940dce82a286 None admin] PATH_INFO: `/v3/services` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:39.412051 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5661bf2-3cf2-48b6-90e7-940dce82a286 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:19:39.412357 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5661bf2-3cf2-48b6-90e7-940dce82a286 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:19:39.413731 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5661bf2-3cf2-48b6-90e7-940dce82a286 None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:19:39.418688 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 506/1012] 10.222.0.49 () {56 vars in 1122 bytes} [Wed Jun 15 10:19:39 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:19:39.424433 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3c06fa-6acc-418b-aaaa-9ec2bcf41e09 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:19:39.526473 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3c06fa-6acc-418b-aaaa-9ec2bcf41e09 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:19:39.537409 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:19:39.563576 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c14a49-52a6-4712-a03a-9c850672b52b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:19:39.564489 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c14a49-52a6-4712-a03a-9c850672b52b None admin] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:39.564926 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c14a49-52a6-4712-a03a-9c850672b52b None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:39.565398 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c14a49-52a6-4712-a03a-9c850672b52b None admin] PATH_INFO: `/v3/endpoints` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:39.566282 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c14a49-52a6-4712-a03a-9c850672b52b None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:19:39.567809 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c14a49-52a6-4712-a03a-9c850672b52b None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:19:39.584607 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 507/1013] 10.222.0.49 () {60 vars in 1144 bytes} [Wed Jun 15 10:19:39 2022] POST /identity/v3/endpoints => generated 325 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 10:19:40.021011 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232d524f-d403-4133-ac9d-cded00f188e1 None None] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:40.021338 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232d524f-d403-4133-ac9d-cded00f188e1 None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:40.021584 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232d524f-d403-4133-ac9d-cded00f188e1 None None] PATH_INFO: `/` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:40.023114 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 507/1014] 10.222.0.49 () {54 vars in 926 bytes} [Wed Jun 15 10:19:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:19:40.028163 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0baf8f48-02c1-481d-b975-003de6d12d5c None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:40.028516 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0baf8f48-02c1-481d-b975-003de6d12d5c None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:40.028861 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0baf8f48-02c1-481d-b975-003de6d12d5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:40.046510 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-0baf8f48-02c1-481d-b975-003de6d12d5c None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:19:40.047713 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-0baf8f48-02c1-481d-b975-003de6d12d5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:19:40.122223 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 508/1015] 10.222.0.49 () {58 vars in 1023 bytes} [Wed Jun 15 10:19:40 2022] POST /identity/v3/auth/tokens => generated 1863 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 15 10:19:40.145775 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232d524f-d403-4133-ac9d-cded00f188e1 None None] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:19:40.156184 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=100 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:19:40.252852 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232d524f-d403-4133-ac9d-cded00f188e1 None None] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:19:40.288188 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48094a1c-c16d-4766-b795-d540f81677c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:19:40.288983 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48094a1c-c16d-4766-b795-d540f81677c8 None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:40.289265 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48094a1c-c16d-4766-b795-d540f81677c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:40.289541 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48094a1c-c16d-4766-b795-d540f81677c8 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:40.301362 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48094a1c-c16d-4766-b795-d540f81677c8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 10:19:40.310041 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application [None req-48094a1c-c16d-4766-b795-d540f81677c8 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 15 10:19:40.310041 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:19:40.310041 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:19:40.310041 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:19:40.310041 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:19:40.310041 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:19:40.310041 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:19:40.310041 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:19:40.310041 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:19:40.310041 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:19:40.310041 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:19:40.310041 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:19:40.310041 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 10:19:40.310041 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 10:19:40.310041 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 10:19:40.310041 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 10:19:40.310952 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 10:19:40.310952 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 10:19:40.310952 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 10:19:40.310952 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 10:19:40.310952 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:19:40.310952 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:19:40.310952 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:19:40.310952 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:19:40.310952 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:19:40.310952 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:19:40.310952 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:19:40.310952 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application with Lock( Jun 15 10:19:40.310952 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:19:40.310952 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:19:40.310952 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:19:40.310952 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:19:40.312105 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:19:40.312105 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:19:40.312105 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:19:40.312105 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:19:40.312105 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 10:19:40.312105 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 10:19:40.312105 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 10:19:40.312105 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 10:19:40.312105 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 10:19:40.312105 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 10:19:40.312105 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 15 10:19:40.312105 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application Jun 15 10:19:40.313163 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 508/1016] 10.222.0.49 () {56 vars in 1120 bytes} [Wed Jun 15 10:19:40 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:19:40.317264 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0baf8f48-02c1-481d-b975-003de6d12d5c None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:19:40.412281 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:19:40.420347 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0baf8f48-02c1-481d-b975-003de6d12d5c None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:19:40.459221 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7376e209-22aa-4452-811d-67fa44c64ff1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:19:40.459900 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7376e209-22aa-4452-811d-67fa44c64ff1 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:40.460183 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7376e209-22aa-4452-811d-67fa44c64ff1 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:40.460456 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7376e209-22aa-4452-811d-67fa44c64ff1 None admin] PATH_INFO: `/v3/roles` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:40.461201 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7376e209-22aa-4452-811d-67fa44c64ff1 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:19:40.461517 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7376e209-22aa-4452-811d-67fa44c64ff1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:19:40.462915 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7376e209-22aa-4452-811d-67fa44c64ff1 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:19:40.475516 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 509/1017] 10.222.0.49 () {56 vars in 1115 bytes} [Wed Jun 15 10:19:40 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 10:19:40.477088 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48094a1c-c16d-4766-b795-d540f81677c8 None admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:19:40.493744 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb8d362-ee52-40d4-8b6a-f53452dc13de None None] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:40.494042 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb8d362-ee52-40d4-8b6a-f53452dc13de None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:40.494323 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb8d362-ee52-40d4-8b6a-f53452dc13de None None] PATH_INFO: `/` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:40.495720 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 510/1018] 10.222.0.49 () {54 vars in 926 bytes} [Wed Jun 15 10:19:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 15 10:19:40.500586 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7b720d-5c14-4319-ab1a-24a8db7a84cf None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:40.500878 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7b720d-5c14-4319-ab1a-24a8db7a84cf None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:40.501161 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7b720d-5c14-4319-ab1a-24a8db7a84cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:40.517826 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-ca7b720d-5c14-4319-ab1a-24a8db7a84cf None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:19:40.519012 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-ca7b720d-5c14-4319-ab1a-24a8db7a84cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:19:40.549904 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=100 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:19:40.585127 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48094a1c-c16d-4766-b795-d540f81677c8 None admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:19:40.589338 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 511/1019] 10.222.0.49 () {58 vars in 1023 bytes} [Wed Jun 15 10:19:40 2022] POST /identity/v3/auth/tokens => generated 1863 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 15 10:19:40.611561 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7b720d-5c14-4319-ab1a-24a8db7a84cf None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:19:40.619796 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9cbe18-d1cc-409a-92c1-46f27570ac6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:19:40.620587 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9cbe18-d1cc-409a-92c1-46f27570ac6d None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:40.620868 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9cbe18-d1cc-409a-92c1-46f27570ac6d None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:40.621138 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9cbe18-d1cc-409a-92c1-46f27570ac6d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:40.631173 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b9cbe18-d1cc-409a-92c1-46f27570ac6d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:19:40.632615 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b9cbe18-d1cc-409a-92c1-46f27570ac6d None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:19:40.651462 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application [None req-9b9cbe18-d1cc-409a-92c1-46f27570ac6d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 10:19:40.651462 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:19:40.651462 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:19:40.651462 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:19:40.651462 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:19:40.651462 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:19:40.651462 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:19:40.651462 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:19:40.651462 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:19:40.651462 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:19:40.651462 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:19:40.651462 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:19:40.651462 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 10:19:40.651462 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 10:19:40.651462 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 10:19:40.652560 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 10:19:40.652560 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:19:40.652560 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:19:40.652560 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:19:40.652560 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:19:40.652560 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:19:40.652560 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:19:40.652560 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:19:40.652560 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application with Lock( Jun 15 10:19:40.652560 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:19:40.652560 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:19:40.652560 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:19:40.652560 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:19:40.652560 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:19:40.652560 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:19:40.652560 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:19:40.652560 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:19:40.653601 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 10:19:40.653601 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 10:19:40.653601 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 10:19:40.653601 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application Jun 15 10:19:40.653601 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 509/1020] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Jun 15 10:19:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 10:19:40.658091 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9cbe18-d1cc-409a-92c1-46f27570ac6d None admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:19:40.706662 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:19:40.720986 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7b720d-5c14-4319-ab1a-24a8db7a84cf None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:19:40.757918 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8961fd77-2fa2-413d-bb4e-fa4340fde479 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:19:40.758582 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8961fd77-2fa2-413d-bb4e-fa4340fde479 None admin] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:40.758900 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8961fd77-2fa2-413d-bb4e-fa4340fde479 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:40.759198 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8961fd77-2fa2-413d-bb4e-fa4340fde479 None admin] PATH_INFO: `/v3/services` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:40.759913 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8961fd77-2fa2-413d-bb4e-fa4340fde479 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:19:40.760445 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9cbe18-d1cc-409a-92c1-46f27570ac6d None admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:19:40.761273 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8961fd77-2fa2-413d-bb4e-fa4340fde479 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:19:40.779679 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 512/1021] 10.222.0.49 () {60 vars in 1141 bytes} [Wed Jun 15 10:19:40 2022] POST /identity/v3/services => generated 242 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 10:19:40.787947 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=100 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:19:40.818374 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9fd0e37-13a2-4a92-9580-3a1c0d78719a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:19:40.819064 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fd0e37-13a2-4a92-9580-3a1c0d78719a None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:40.819377 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fd0e37-13a2-4a92-9580-3a1c0d78719a None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:40.819650 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fd0e37-13a2-4a92-9580-3a1c0d78719a None admin] PATH_INFO: `/v3/domains` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:40.820378 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9fd0e37-13a2-4a92-9580-3a1c0d78719a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:19:40.820681 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9fd0e37-13a2-4a92-9580-3a1c0d78719a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:19:40.822118 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9fd0e37-13a2-4a92-9580-3a1c0d78719a None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:19:40.841099 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 510/1022] 10.222.0.49 () {56 vars in 1109 bytes} [Wed Jun 15 10:19:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 10:19:40.842992 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8961fd77-2fa2-413d-bb4e-fa4340fde479 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:19:40.954539 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8961fd77-2fa2-413d-bb4e-fa4340fde479 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:19:40.990493 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6115c5a0-9c1d-43c2-b7af-0f55585fdf80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:19:40.991317 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6115c5a0-9c1d-43c2-b7af-0f55585fdf80 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:40.991601 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6115c5a0-9c1d-43c2-b7af-0f55585fdf80 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:40.991876 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6115c5a0-9c1d-43c2-b7af-0f55585fdf80 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:41.000304 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6115c5a0-9c1d-43c2-b7af-0f55585fdf80 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:19:41.001753 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6115c5a0-9c1d-43c2-b7af-0f55585fdf80 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:19:41.015623 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application [None req-6115c5a0-9c1d-43c2-b7af-0f55585fdf80 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 15 10:19:41.015623 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:19:41.015623 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:19:41.015623 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:19:41.015623 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:19:41.015623 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:19:41.015623 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:19:41.015623 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:19:41.015623 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:19:41.015623 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:19:41.015623 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:19:41.015623 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:19:41.015623 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 15 10:19:41.015623 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 15 10:19:41.015623 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 15 10:19:41.015623 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 15 10:19:41.016777 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:19:41.016777 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:19:41.016777 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 15 10:19:41.016777 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 10:19:41.016777 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 15 10:19:41.016777 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 10:19:41.016777 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:19:41.016777 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:19:41.016777 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:19:41.016777 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:19:41.016777 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:19:41.016777 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application with Lock( Jun 15 10:19:41.016777 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:19:41.016777 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:19:41.016777 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:19:41.016777 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:19:41.016777 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:19:41.016777 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:19:41.017841 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:19:41.017841 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:19:41.017841 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 15 10:19:41.017841 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 15 10:19:41.017841 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 15 10:19:41.017841 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 15 10:19:41.017841 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 15 10:19:41.017841 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 15 10:19:41.017841 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 15 10:19:41.017841 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Jun 15 10:19:41.017841 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 513/1023] 10.222.0.49 () {56 vars in 1093 bytes} [Wed Jun 15 10:19:40 2022] GET /identity/v3/users/nova => generated 82 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:19:41.023588 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9fd0e37-13a2-4a92-9580-3a1c0d78719a None admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:19:41.127190 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9fd0e37-13a2-4a92-9580-3a1c0d78719a None admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:19:41.161005 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d970c6-4999-4ae2-baf6-e32d6df1a09a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:19:41.161671 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d970c6-4999-4ae2-baf6-e32d6df1a09a None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:41.161939 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d970c6-4999-4ae2-baf6-e32d6df1a09a None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:41.162215 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d970c6-4999-4ae2-baf6-e32d6df1a09a None admin] PATH_INFO: `/v3/users` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:41.163044 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d970c6-4999-4ae2-baf6-e32d6df1a09a None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:19:41.163418 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d970c6-4999-4ae2-baf6-e32d6df1a09a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:19:41.164951 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d970c6-4999-4ae2-baf6-e32d6df1a09a None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:19:41.188136 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 511/1024] 10.222.0.49 () {56 vars in 1133 bytes} [Wed Jun 15 10:19:41 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:19:41.189466 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6115c5a0-9c1d-43c2-b7af-0f55585fdf80 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:19:41.201559 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:19:41.297997 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6115c5a0-9c1d-43c2-b7af-0f55585fdf80 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:19:41.330996 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed18203-2afb-4ea0-bb7f-77bfeff4743d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:19:41.331773 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed18203-2afb-4ea0-bb7f-77bfeff4743d None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:41.331956 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed18203-2afb-4ea0-bb7f-77bfeff4743d None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:41.332233 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed18203-2afb-4ea0-bb7f-77bfeff4743d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:41.401455 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed18203-2afb-4ea0-bb7f-77bfeff4743d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:19:41.402828 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed18203-2afb-4ea0-bb7f-77bfeff4743d None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:19:41.412312 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:19:41.505116 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 514/1025] 10.222.0.49 () {58 vars in 1282 bytes} [Wed Jun 15 10:19:41 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 15 10:19:41.506605 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d970c6-4999-4ae2-baf6-e32d6df1a09a None admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:19:41.539439 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=100 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:19:41.609463 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d970c6-4999-4ae2-baf6-e32d6df1a09a None admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:19:41.642772 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53e911e-f10a-4099-95fa-4a19cb64431c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:19:41.643591 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53e911e-f10a-4099-95fa-4a19cb64431c None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:41.643877 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53e911e-f10a-4099-95fa-4a19cb64431c None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:41.644152 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53e911e-f10a-4099-95fa-4a19cb64431c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:41.654808 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53e911e-f10a-4099-95fa-4a19cb64431c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:19:41.656226 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53e911e-f10a-4099-95fa-4a19cb64431c None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:19:41.675034 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application [None req-e53e911e-f10a-4099-95fa-4a19cb64431c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 10:19:41.675034 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:19:41.675034 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:19:41.675034 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:19:41.675034 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:19:41.675034 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:19:41.675034 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:19:41.675034 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:19:41.675034 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:19:41.675034 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:19:41.675034 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:19:41.675034 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:19:41.675034 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 10:19:41.675034 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 10:19:41.675034 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 10:19:41.676053 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 10:19:41.676053 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:19:41.676053 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:19:41.676053 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:19:41.676053 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:19:41.676053 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:19:41.676053 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:19:41.676053 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:19:41.676053 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application with Lock( Jun 15 10:19:41.676053 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:19:41.676053 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:19:41.676053 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:19:41.676053 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:19:41.676053 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:19:41.676053 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:19:41.676053 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:19:41.676053 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:19:41.676973 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 10:19:41.676973 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 10:19:41.676973 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 10:19:41.676973 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application Jun 15 10:19:41.676973 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 512/1026] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Jun 15 10:19:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 10:19:41.683243 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed18203-2afb-4ea0-bb7f-77bfeff4743d None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:19:41.733357 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada554e0-5b3b-4279-b173-1d19e8091f59 None None] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:41.733678 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada554e0-5b3b-4279-b173-1d19e8091f59 None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:41.733989 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada554e0-5b3b-4279-b173-1d19e8091f59 None None] PATH_INFO: `/` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:41.735444 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 513/1027] 10.222.0.49 () {54 vars in 926 bytes} [Wed Jun 15 10:19:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 15 10:19:41.740051 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a608c5-ccf8-4ee5-9f9f-63d0fb04a9db None None] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:41.740345 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a608c5-ccf8-4ee5-9f9f-63d0fb04a9db None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:41.740627 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a608c5-ccf8-4ee5-9f9f-63d0fb04a9db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:41.756952 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.auth.core [None req-37a608c5-ccf8-4ee5-9f9f-63d0fb04a9db None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:19:41.758157 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-37a608c5-ccf8-4ee5-9f9f-63d0fb04a9db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:19:41.790648 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed18203-2afb-4ea0-bb7f-77bfeff4743d None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:19:41.800959 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:19:41.828895 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77e438e-e8b2-438a-9433-3dfa28c666f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:19:41.829563 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77e438e-e8b2-438a-9433-3dfa28c666f3 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:41.829848 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77e438e-e8b2-438a-9433-3dfa28c666f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:41.830124 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77e438e-e8b2-438a-9433-3dfa28c666f3 None admin] PATH_INFO: `/v3/domains` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:41.830847 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a77e438e-e8b2-438a-9433-3dfa28c666f3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:19:41.831179 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a77e438e-e8b2-438a-9433-3dfa28c666f3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:19:41.832550 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a77e438e-e8b2-438a-9433-3dfa28c666f3 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:19:41.840841 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 514/1028] 10.222.0.49 () {58 vars in 1023 bytes} [Wed Jun 15 10:19:41 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 15 10:19:41.844249 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 515/1029] 10.222.0.49 () {56 vars in 1109 bytes} [Wed Jun 15 10:19:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:19:41.849760 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a608c5-ccf8-4ee5-9f9f-63d0fb04a9db None None] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:19:41.864356 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77e438e-e8b2-438a-9433-3dfa28c666f3 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:19:41.864852 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=100 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:19:41.959175 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a608c5-ccf8-4ee5-9f9f-63d0fb04a9db None None] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:19:41.970744 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77e438e-e8b2-438a-9433-3dfa28c666f3 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:19:41.995788 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6faaca-b5f0-4bd0-a06b-6cd352a99cb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:19:41.996596 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6faaca-b5f0-4bd0-a06b-6cd352a99cb5 None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:41.996883 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6faaca-b5f0-4bd0-a06b-6cd352a99cb5 None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:41.997161 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6faaca-b5f0-4bd0-a06b-6cd352a99cb5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:42.004256 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b6faaca-b5f0-4bd0-a06b-6cd352a99cb5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:19:42.005719 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b6faaca-b5f0-4bd0-a06b-6cd352a99cb5 None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:19:42.005992 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a360c51e-785a-4a1a-bc4d-9f61ffc6bd34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:19:42.006788 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a360c51e-785a-4a1a-bc4d-9f61ffc6bd34 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:42.007074 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a360c51e-785a-4a1a-bc4d-9f61ffc6bd34 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:42.007379 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a360c51e-785a-4a1a-bc4d-9f61ffc6bd34 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:42.008561 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a360c51e-785a-4a1a-bc4d-9f61ffc6bd34 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:19:42.010527 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a360c51e-785a-4a1a-bc4d-9f61ffc6bd34 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:19:42.012337 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 516/1030] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Jun 15 10:19:41 2022] GET /identity/v3/domains/default => generated 199 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 10:19:42.017704 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a360c51e-785a-4a1a-bc4d-9f61ffc6bd34 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:19:42.020795 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application [None req-0b6faaca-b5f0-4bd0-a06b-6cd352a99cb5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 10:19:42.020795 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:19:42.020795 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:19:42.020795 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:19:42.020795 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:19:42.020795 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:19:42.020795 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:19:42.020795 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:19:42.020795 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:19:42.020795 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:19:42.020795 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:19:42.020795 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:19:42.020795 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 15 10:19:42.020795 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 15 10:19:42.020795 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 15 10:19:42.021723 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 15 10:19:42.021723 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:19:42.021723 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:19:42.021723 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:19:42.021723 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:19:42.021723 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:19:42.021723 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:19:42.021723 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:19:42.021723 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application with Lock( Jun 15 10:19:42.021723 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:19:42.021723 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:19:42.021723 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:19:42.021723 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:19:42.021723 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:19:42.021723 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:19:42.021723 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:19:42.021723 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:19:42.022667 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 15 10:19:42.022667 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 15 10:19:42.022667 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 15 10:19:42.022667 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 15 10:19:42.022667 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 15 10:19:42.022667 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 15 10:19:42.022667 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 10:19:42.022667 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application Jun 15 10:19:42.022667 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 515/1031] 10.222.0.49 () {56 vars in 1111 bytes} [Wed Jun 15 10:19:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 10:19:42.028792 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6faaca-b5f0-4bd0-a06b-6cd352a99cb5 None admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:19:42.061393 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:19:42.107964 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=100 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:19:42.127856 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a360c51e-785a-4a1a-bc4d-9f61ffc6bd34 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:19:42.136282 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6faaca-b5f0-4bd0-a06b-6cd352a99cb5 None admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:19:42.164961 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aefdec58-50d9-46ea-a539-3af9aa52d7fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:19:42.165627 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefdec58-50d9-46ea-a539-3af9aa52d7fe None admin] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:42.165916 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefdec58-50d9-46ea-a539-3af9aa52d7fe None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:42.166195 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefdec58-50d9-46ea-a539-3af9aa52d7fe None admin] PATH_INFO: `/v3/projects` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:42.166898 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aefdec58-50d9-46ea-a539-3af9aa52d7fe None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:19:42.168363 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aefdec58-50d9-46ea-a539-3af9aa52d7fe None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:19:42.171432 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f442807-e34a-4c5e-bf36-e43e9656a35a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:19:42.172082 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f442807-e34a-4c5e-bf36-e43e9656a35a None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:42.172363 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f442807-e34a-4c5e-bf36-e43e9656a35a None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:42.172641 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f442807-e34a-4c5e-bf36-e43e9656a35a None admin] PATH_INFO: `/v3/projects` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:42.173414 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f442807-e34a-4c5e-bf36-e43e9656a35a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:19:42.173720 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f442807-e34a-4c5e-bf36-e43e9656a35a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:19:42.175412 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f442807-e34a-4c5e-bf36-e43e9656a35a None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:19:42.193984 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 516/1032] 10.222.0.49 () {56 vars in 1148 bytes} [Wed Jun 15 10:19:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 10:19:42.194714 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f442807-e34a-4c5e-bf36-e43e9656a35a None admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:19:42.207369 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 517/1033] 10.222.0.49 () {60 vars in 1141 bytes} [Wed Jun 15 10:19:42 2022] POST /identity/v3/projects => generated 311 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 10:19:42.315527 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f442807-e34a-4c5e-bf36-e43e9656a35a None admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:19:42.353553 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea76cae-d09c-411e-a3e3-1081436bc779 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:19:42.354225 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea76cae-d09c-411e-a3e3-1081436bc779 None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:42.354512 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea76cae-d09c-411e-a3e3-1081436bc779 None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:42.355307 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea76cae-d09c-411e-a3e3-1081436bc779 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:42.355660 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea76cae-d09c-411e-a3e3-1081436bc779 None admin] RBAC: Adding query filter params (role.id=d962cdb5fdb24466a560f5d7bcb1e034, scope.project.id=559b3e713f5844b6ae37151d56dd419f, user.id=5ac54528370d4747b6e8777fba9d0c82) {{(pid=80119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:19:42.356622 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea76cae-d09c-411e-a3e3-1081436bc779 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:19:42.357743 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea76cae-d09c-411e-a3e3-1081436bc779 None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:19:42.359780 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=100 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:19:42.366661 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 517/1034] 10.222.0.49 () {56 vars in 1374 bytes} [Wed Jun 15 10:19:42 2022] GET /identity/v3/role_assignments?user.id=5ac54528370d4747b6e8777fba9d0c82&scope.project.id=559b3e713f5844b6ae37151d56dd419f&role.id=d962cdb5fdb24466a560f5d7bcb1e034 => generated 259 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:19:42.940140 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f59c9b-b721-47ab-8d4f-ca6c27baf5e8 None None] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:42.940676 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f59c9b-b721-47ab-8d4f-ca6c27baf5e8 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:42.940676 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f59c9b-b721-47ab-8d4f-ca6c27baf5e8 None None] PATH_INFO: `/` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:42.942217 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 518/1035] 10.222.0.49 () {54 vars in 926 bytes} [Wed Jun 15 10:19:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:19:42.947869 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea11525-b321-4734-8c49-6b2bfa610285 None None] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:42.948162 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea11525-b321-4734-8c49-6b2bfa610285 None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:42.948443 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea11525-b321-4734-8c49-6b2bfa610285 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:42.965514 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.auth.core [None req-9ea11525-b321-4734-8c49-6b2bfa610285 None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:19:42.966724 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-9ea11525-b321-4734-8c49-6b2bfa610285 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:19:43.035986 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 518/1036] 10.222.0.49 () {58 vars in 1023 bytes} [Wed Jun 15 10:19:42 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 10:19:43.059509 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f59c9b-b721-47ab-8d4f-ca6c27baf5e8 None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:19:43.151595 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:19:43.166605 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f59c9b-b721-47ab-8d4f-ca6c27baf5e8 None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:19:43.201101 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb698228-b0da-4a5b-8ed7-8e500466c206 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:19:43.202203 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb698228-b0da-4a5b-8ed7-8e500466c206 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:43.202710 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb698228-b0da-4a5b-8ed7-8e500466c206 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:43.203232 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb698228-b0da-4a5b-8ed7-8e500466c206 None admin] PATH_INFO: `/v3/services/image` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:43.204097 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb698228-b0da-4a5b-8ed7-8e500466c206 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:19:43.205667 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb698228-b0da-4a5b-8ed7-8e500466c206 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:19:43.217844 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application [None req-cb698228-b0da-4a5b-8ed7-8e500466c206 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jun 15 10:19:43.217844 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:19:43.217844 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 15 10:19:43.217844 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 15 10:19:43.217844 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 15 10:19:43.217844 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 15 10:19:43.217844 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 15 10:19:43.217844 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 10:19:43.217844 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 15 10:19:43.217844 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Jun 15 10:19:43.217844 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 15 10:19:43.217844 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Jun 15 10:19:43.217844 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:19:43.217844 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:19:43.217844 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:19:43.217844 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:19:43.217844 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:19:43.218889 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:19:43.218889 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:19:43.218889 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:19:43.218889 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:19:43.218889 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:19:43.218889 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:19:43.218889 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 15 10:19:43.218889 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 15 10:19:43.218889 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 15 10:19:43.218889 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 15 10:19:43.218889 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:19:43.218889 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:19:43.218889 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:19:43.218889 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:19:43.218889 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:19:43.218889 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:19:43.220167 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:19:43.220167 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application with Lock( Jun 15 10:19:43.220167 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:19:43.220167 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:19:43.220167 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:19:43.220167 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:19:43.220167 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:19:43.220167 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:19:43.220167 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:19:43.220167 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:19:43.220167 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 15 10:19:43.220167 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 10:19:43.220167 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 15 10:19:43.220167 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Jun 15 10:19:43.223234 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 519/1037] 10.222.0.49 () {56 vars in 1105 bytes} [Wed Jun 15 10:19:43 2022] GET /identity/v3/services/image => generated 86 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:19:43.225892 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea11525-b321-4734-8c49-6b2bfa610285 None None] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:19:43.325787 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea11525-b321-4734-8c49-6b2bfa610285 None None] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:19:43.354373 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=100 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:19:43.364744 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f47c479-a101-4bb5-8965-1a44c241b9ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:19:43.365434 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f47c479-a101-4bb5-8965-1a44c241b9ae None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:43.365722 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f47c479-a101-4bb5-8965-1a44c241b9ae None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:43.366002 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f47c479-a101-4bb5-8965-1a44c241b9ae None admin] PATH_INFO: `/v3/services` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:43.366709 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f47c479-a101-4bb5-8965-1a44c241b9ae None admin] RBAC: Adding query filter params (name=image) {{(pid=80119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:19:43.367022 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f47c479-a101-4bb5-8965-1a44c241b9ae None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:19:43.368972 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f47c479-a101-4bb5-8965-1a44c241b9ae None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:19:43.378612 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 519/1038] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Jun 15 10:19:43 2022] GET /identity/v3/services?name=image => generated 122 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 10:19:43.380165 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb698228-b0da-4a5b-8ed7-8e500466c206 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:19:43.483834 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb698228-b0da-4a5b-8ed7-8e500466c206 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:19:43.518129 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91903c8e-3e7c-4e47-810d-3386d2670b9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:19:43.518799 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91903c8e-3e7c-4e47-810d-3386d2670b9e None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:43.519104 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91903c8e-3e7c-4e47-810d-3386d2670b9e None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:43.519389 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91903c8e-3e7c-4e47-810d-3386d2670b9e None admin] PATH_INFO: `/v3/services` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:43.520302 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91903c8e-3e7c-4e47-810d-3386d2670b9e None admin] RBAC: Adding query filter params (type=image) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:19:43.520664 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91903c8e-3e7c-4e47-810d-3386d2670b9e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:19:43.522306 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91903c8e-3e7c-4e47-810d-3386d2670b9e None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:19:43.532172 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 520/1039] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Jun 15 10:19:43 2022] GET /identity/v3/services?type=image => generated 350 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 10:19:43.533817 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f47c479-a101-4bb5-8965-1a44c241b9ae None admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:19:43.634144 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f47c479-a101-4bb5-8965-1a44c241b9ae None admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:19:43.668060 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b2da4b-4499-4bea-a9e7-dfc98152c4fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:19:43.668718 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b2da4b-4499-4bea-a9e7-dfc98152c4fe None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:43.669004 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b2da4b-4499-4bea-a9e7-dfc98152c4fe None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:43.669283 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b2da4b-4499-4bea-a9e7-dfc98152c4fe None admin] PATH_INFO: `/v3/endpoints` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:43.670065 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b2da4b-4499-4bea-a9e7-dfc98152c4fe None admin] RBAC: Adding query filter params (interface=public, service_id=1230db23ba844072ad6a6f891db62221, region_id=RegionOne) {{(pid=80119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:19:43.670378 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b2da4b-4499-4bea-a9e7-dfc98152c4fe None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:19:43.671783 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b2da4b-4499-4bea-a9e7-dfc98152c4fe None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:19:43.677298 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 520/1040] 10.222.0.49 () {56 vars in 1251 bytes} [Wed Jun 15 10:19:43 2022] GET /identity/v3/endpoints?service_id=1230db23ba844072ad6a6f891db62221&interface=public®ion_id=RegionOne => generated 194 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 10:19:43.683800 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91903c8e-3e7c-4e47-810d-3386d2670b9e None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:19:43.702282 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:19:43.787762 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91903c8e-3e7c-4e47-810d-3386d2670b9e None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:19:43.825661 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe23f070-d93c-4b1e-b063-3fd0fb4d233e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:19:43.826335 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe23f070-d93c-4b1e-b063-3fd0fb4d233e None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:43.826622 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe23f070-d93c-4b1e-b063-3fd0fb4d233e None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:43.826899 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe23f070-d93c-4b1e-b063-3fd0fb4d233e None admin] PATH_INFO: `/v3/services` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:43.827614 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe23f070-d93c-4b1e-b063-3fd0fb4d233e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:19:43.828988 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe23f070-d93c-4b1e-b063-3fd0fb4d233e None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:19:43.843408 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 521/1041] 10.222.0.49 () {56 vars in 1087 bytes} [Wed Jun 15 10:19:43 2022] GET /identity/v3/services => generated 1957 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 10:19:44.379491 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c82ccd4-e9cb-4a64-abb6-82e23c1c6bc5 None None] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:44.380012 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c82ccd4-e9cb-4a64-abb6-82e23c1c6bc5 None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:44.380089 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c82ccd4-e9cb-4a64-abb6-82e23c1c6bc5 None None] PATH_INFO: `/` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:44.381395 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 521/1042] 10.222.0.49 () {54 vars in 926 bytes} [Wed Jun 15 10:19:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:19:44.387623 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468d54e0-cb79-47e5-8728-13bde1f97ab8 None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:44.388069 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468d54e0-cb79-47e5-8728-13bde1f97ab8 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:44.388187 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468d54e0-cb79-47e5-8728-13bde1f97ab8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:44.406970 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-468d54e0-cb79-47e5-8728-13bde1f97ab8 None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:19:44.408262 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-468d54e0-cb79-47e5-8728-13bde1f97ab8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:19:44.465774 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232ce8a8-e93a-4a20-b284-2c469ae83298 None None] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:44.466054 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232ce8a8-e93a-4a20-b284-2c469ae83298 None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:44.466506 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232ce8a8-e93a-4a20-b284-2c469ae83298 None None] PATH_INFO: `/` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:44.467889 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 522/1043] 10.222.0.49 () {54 vars in 926 bytes} [Wed Jun 15 10:19:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 15 10:19:44.473064 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe62ff63-1a9a-400b-8d8b-f0184828a531 None None] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:44.473532 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe62ff63-1a9a-400b-8d8b-f0184828a531 None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:44.473819 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe62ff63-1a9a-400b-8d8b-f0184828a531 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:44.483438 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 522/1044] 10.222.0.49 () {58 vars in 1023 bytes} [Wed Jun 15 10:19:44 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 10:19:44.491219 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.auth.core [None req-fe62ff63-1a9a-400b-8d8b-f0184828a531 None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:19:44.492423 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-fe62ff63-1a9a-400b-8d8b-f0184828a531 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:19:44.507488 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468d54e0-cb79-47e5-8728-13bde1f97ab8 None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:19:44.520600 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:19:44.548875 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=100 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:19:44.568557 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 523/1045] 10.222.0.49 () {58 vars in 1023 bytes} [Wed Jun 15 10:19:44 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 15 10:19:44.591539 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe62ff63-1a9a-400b-8d8b-f0184828a531 None None] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:19:44.618208 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468d54e0-cb79-47e5-8728-13bde1f97ab8 None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:19:44.656769 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55050d4-bef5-4c4d-af73-4bc8893ffdea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:19:44.657480 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55050d4-bef5-4c4d-af73-4bc8893ffdea None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:44.657760 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55050d4-bef5-4c4d-af73-4bc8893ffdea None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:44.658029 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55050d4-bef5-4c4d-af73-4bc8893ffdea None admin] PATH_INFO: `/v3/domains/default` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:44.659219 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d55050d4-bef5-4c4d-af73-4bc8893ffdea None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:19:44.660639 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d55050d4-bef5-4c4d-af73-4bc8893ffdea None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:19:44.662251 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 523/1046] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Jun 15 10:19:44 2022] GET /identity/v3/domains/default => generated 199 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 10:19:44.667759 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55050d4-bef5-4c4d-af73-4bc8893ffdea None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:19:44.696894 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe62ff63-1a9a-400b-8d8b-f0184828a531 None None] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:19:44.737655 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fec41d1-eb39-42d0-8041-4dfa3587b1fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:19:44.738477 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fec41d1-eb39-42d0-8041-4dfa3587b1fb None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:44.738923 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fec41d1-eb39-42d0-8041-4dfa3587b1fb None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:44.739250 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fec41d1-eb39-42d0-8041-4dfa3587b1fb None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:44.751025 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:19:44.752412 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fec41d1-eb39-42d0-8041-4dfa3587b1fb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 10:19:44.761544 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application [None req-3fec41d1-eb39-42d0-8041-4dfa3587b1fb None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 15 10:19:44.761544 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:19:44.761544 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:19:44.761544 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:19:44.761544 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:19:44.761544 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:19:44.761544 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:19:44.761544 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:19:44.761544 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:19:44.761544 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:19:44.761544 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:19:44.761544 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:19:44.761544 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 10:19:44.761544 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 10:19:44.761544 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 10:19:44.761544 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 10:19:44.762493 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 10:19:44.762493 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 10:19:44.762493 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 10:19:44.762493 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 10:19:44.762493 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:19:44.762493 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:19:44.762493 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:19:44.762493 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:19:44.762493 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:19:44.762493 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:19:44.762493 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:19:44.762493 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application with Lock( Jun 15 10:19:44.762493 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:19:44.762493 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:19:44.762493 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:19:44.762493 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:19:44.763724 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:19:44.763724 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:19:44.763724 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:19:44.763724 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:19:44.763724 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 10:19:44.763724 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 10:19:44.763724 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 10:19:44.763724 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 10:19:44.763724 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 10:19:44.763724 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 10:19:44.763724 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 15 10:19:44.763724 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application Jun 15 10:19:44.763724 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 524/1047] 10.222.0.49 () {56 vars in 1120 bytes} [Wed Jun 15 10:19:44 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 10:19:44.769036 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fec41d1-eb39-42d0-8041-4dfa3587b1fb None admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:19:44.778637 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55050d4-bef5-4c4d-af73-4bc8893ffdea None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:19:44.819945 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e61adfc-d7bb-445d-8509-a943d5f93aba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:19:44.820727 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e61adfc-d7bb-445d-8509-a943d5f93aba None admin] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:44.821069 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e61adfc-d7bb-445d-8509-a943d5f93aba None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:44.821832 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e61adfc-d7bb-445d-8509-a943d5f93aba None admin] PATH_INFO: `/v3/users` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:44.822530 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e61adfc-d7bb-445d-8509-a943d5f93aba None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:19:44.824125 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e61adfc-d7bb-445d-8509-a943d5f93aba None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:19:44.824583 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=100 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:19:44.846718 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 524/1048] 10.222.0.49 () {60 vars in 1132 bytes} [Wed Jun 15 10:19:44 2022] POST /identity/v3/users => generated 285 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 10:19:44.886556 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fec41d1-eb39-42d0-8041-4dfa3587b1fb None admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:19:44.925531 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4164bfd3-f28b-4026-9000-d6921fe3e91e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:19:44.926137 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4164bfd3-f28b-4026-9000-d6921fe3e91e None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:44.926417 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4164bfd3-f28b-4026-9000-d6921fe3e91e None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:44.926693 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4164bfd3-f28b-4026-9000-d6921fe3e91e None admin] PATH_INFO: `/v3/roles` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:44.927473 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4164bfd3-f28b-4026-9000-d6921fe3e91e None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:19:44.927781 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4164bfd3-f28b-4026-9000-d6921fe3e91e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:19:44.929183 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4164bfd3-f28b-4026-9000-d6921fe3e91e None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:19:44.943393 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 525/1049] 10.222.0.49 () {56 vars in 1115 bytes} [Wed Jun 15 10:19:44 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 10:19:44.944961 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e61adfc-d7bb-445d-8509-a943d5f93aba None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:19:45.049753 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e61adfc-d7bb-445d-8509-a943d5f93aba None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:19:45.083449 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9463580-77be-4a5c-ba1a-a23a531e8141 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:19:45.084249 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9463580-77be-4a5c-ba1a-a23a531e8141 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:45.084531 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9463580-77be-4a5c-ba1a-a23a531e8141 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:45.084806 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9463580-77be-4a5c-ba1a-a23a531e8141 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:45.086765 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:19:45.098738 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9463580-77be-4a5c-ba1a-a23a531e8141 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:19:45.100236 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9463580-77be-4a5c-ba1a-a23a531e8141 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:19:45.117336 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application [None req-c9463580-77be-4a5c-ba1a-a23a531e8141 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 10:19:45.117336 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:19:45.117336 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:19:45.117336 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:19:45.117336 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:19:45.117336 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:19:45.117336 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:19:45.117336 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:19:45.117336 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:19:45.117336 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:19:45.117336 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:19:45.117336 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:19:45.117336 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 10:19:45.117336 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 10:19:45.117336 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 10:19:45.118775 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 10:19:45.118775 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:19:45.118775 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:19:45.118775 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:19:45.118775 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:19:45.118775 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:19:45.118775 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:19:45.118775 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:19:45.118775 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application with Lock( Jun 15 10:19:45.118775 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:19:45.118775 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:19:45.118775 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:19:45.118775 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:19:45.118775 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:19:45.118775 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:19:45.118775 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:19:45.118775 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:19:45.120038 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 10:19:45.120038 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 10:19:45.120038 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 10:19:45.120038 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Jun 15 10:19:45.120038 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 525/1050] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Jun 15 10:19:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:19:45.124854 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4164bfd3-f28b-4026-9000-d6921fe3e91e None admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:19:45.225564 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4164bfd3-f28b-4026-9000-d6921fe3e91e None admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:19:45.234936 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=100 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:19:45.260530 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0604a6-2577-4139-b236-2fe585566908 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:19:45.261196 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0604a6-2577-4139-b236-2fe585566908 None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:45.261474 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0604a6-2577-4139-b236-2fe585566908 None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:45.261750 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0604a6-2577-4139-b236-2fe585566908 None admin] PATH_INFO: `/v3/domains` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:45.262460 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0604a6-2577-4139-b236-2fe585566908 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:19:45.262763 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0604a6-2577-4139-b236-2fe585566908 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:19:45.264161 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0604a6-2577-4139-b236-2fe585566908 None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:19:45.276042 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 526/1051] 10.222.0.49 () {56 vars in 1109 bytes} [Wed Jun 15 10:19:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:19:45.281410 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9463580-77be-4a5c-ba1a-a23a531e8141 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:19:45.381394 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9463580-77be-4a5c-ba1a-a23a531e8141 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:19:45.414433 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f44e5b-fe10-488e-83ef-74463b3f95e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:19:45.415768 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f44e5b-fe10-488e-83ef-74463b3f95e6 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:45.416049 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f44e5b-fe10-488e-83ef-74463b3f95e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:45.416323 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f44e5b-fe10-488e-83ef-74463b3f95e6 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:45.424295 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f44e5b-fe10-488e-83ef-74463b3f95e6 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:19:45.425735 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f44e5b-fe10-488e-83ef-74463b3f95e6 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:19:45.440315 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application [None req-13f44e5b-fe10-488e-83ef-74463b3f95e6 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 15 10:19:45.440315 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:19:45.440315 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:19:45.440315 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:19:45.440315 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:19:45.440315 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:19:45.440315 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:19:45.440315 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:19:45.440315 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:19:45.440315 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:19:45.440315 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:19:45.440315 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:19:45.440315 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 15 10:19:45.440315 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 15 10:19:45.440315 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 15 10:19:45.440315 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 15 10:19:45.441272 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:19:45.441272 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:19:45.441272 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 15 10:19:45.441272 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 10:19:45.441272 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 15 10:19:45.441272 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 10:19:45.441272 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:19:45.441272 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:19:45.441272 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:19:45.441272 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:19:45.441272 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:19:45.441272 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application with Lock( Jun 15 10:19:45.441272 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:19:45.441272 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:19:45.441272 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:19:45.441272 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:19:45.441272 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:19:45.441272 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:19:45.442233 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:19:45.442233 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:19:45.442233 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 15 10:19:45.442233 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 15 10:19:45.442233 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 15 10:19:45.442233 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 15 10:19:45.442233 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 15 10:19:45.442233 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 15 10:19:45.442233 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 15 10:19:45.442233 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Jun 15 10:19:45.447130 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 526/1052] 10.222.0.49 () {56 vars in 1093 bytes} [Wed Jun 15 10:19:45 2022] GET /identity/v3/users/nova => generated 82 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:19:45.447405 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0604a6-2577-4139-b236-2fe585566908 None admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:19:45.550768 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0604a6-2577-4139-b236-2fe585566908 None admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:19:45.584541 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e05865-9688-41cf-ba76-e20dc2d3d5d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:19:45.585221 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e05865-9688-41cf-ba76-e20dc2d3d5d1 None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:45.585505 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e05865-9688-41cf-ba76-e20dc2d3d5d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:45.585781 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e05865-9688-41cf-ba76-e20dc2d3d5d1 None admin] PATH_INFO: `/v3/users` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:45.587158 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e05865-9688-41cf-ba76-e20dc2d3d5d1 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:19:45.587473 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e05865-9688-41cf-ba76-e20dc2d3d5d1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:19:45.588982 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e05865-9688-41cf-ba76-e20dc2d3d5d1 None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:19:45.612140 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 527/1053] 10.222.0.49 () {56 vars in 1133 bytes} [Wed Jun 15 10:19:45 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:19:45.613514 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f44e5b-fe10-488e-83ef-74463b3f95e6 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:19:45.661917 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:19:45.717828 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f44e5b-fe10-488e-83ef-74463b3f95e6 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:19:45.751570 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e933da56-9481-42ac-884d-40164e56c675 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:19:45.752362 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e933da56-9481-42ac-884d-40164e56c675 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:45.752641 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e933da56-9481-42ac-884d-40164e56c675 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:45.752913 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e933da56-9481-42ac-884d-40164e56c675 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:45.763357 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e933da56-9481-42ac-884d-40164e56c675 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:19:45.764822 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e933da56-9481-42ac-884d-40164e56c675 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:19:45.783705 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application [None req-e933da56-9481-42ac-884d-40164e56c675 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 10:19:45.783705 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:19:45.783705 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:19:45.783705 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:19:45.783705 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:19:45.783705 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:19:45.783705 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:19:45.783705 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:19:45.783705 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:19:45.783705 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:19:45.783705 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:19:45.783705 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:19:45.783705 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 10:19:45.783705 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 10:19:45.783705 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 10:19:45.784655 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 10:19:45.784655 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:19:45.784655 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:19:45.784655 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:19:45.784655 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:19:45.784655 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:19:45.784655 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:19:45.784655 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:19:45.784655 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application with Lock( Jun 15 10:19:45.784655 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:19:45.784655 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:19:45.784655 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:19:45.784655 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:19:45.784655 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:19:45.784655 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:19:45.784655 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:19:45.784655 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:19:45.785565 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 10:19:45.785565 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 10:19:45.785565 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 10:19:45.785565 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Jun 15 10:19:45.787410 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 527/1054] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Jun 15 10:19:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:19:45.792294 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e05865-9688-41cf-ba76-e20dc2d3d5d1 None admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:19:45.819930 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=100 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:19:45.894325 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e05865-9688-41cf-ba76-e20dc2d3d5d1 None admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:19:45.932125 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9433f0df-516b-4193-9ce5-242149fa07d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:19:45.932795 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9433f0df-516b-4193-9ce5-242149fa07d2 None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:45.933075 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9433f0df-516b-4193-9ce5-242149fa07d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:45.933620 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9433f0df-516b-4193-9ce5-242149fa07d2 None admin] PATH_INFO: `/v3/domains` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:45.934349 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9433f0df-516b-4193-9ce5-242149fa07d2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:19:45.934651 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9433f0df-516b-4193-9ce5-242149fa07d2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:19:45.936046 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9433f0df-516b-4193-9ce5-242149fa07d2 None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:19:45.952346 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 528/1055] 10.222.0.49 () {56 vars in 1109 bytes} [Wed Jun 15 10:19:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:19:45.953492 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e933da56-9481-42ac-884d-40164e56c675 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:19:45.967240 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6082335-caa6-4db1-a116-5c8870a85485 None None] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:45.967540 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6082335-caa6-4db1-a116-5c8870a85485 None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:45.967827 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6082335-caa6-4db1-a116-5c8870a85485 None None] PATH_INFO: `/` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:45.969226 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 529/1056] 10.222.0.49 () {54 vars in 926 bytes} [Wed Jun 15 10:19:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 15 10:19:45.974010 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511d4b5c-a4c5-4f3b-8918-bd41a00104cc None None] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:45.974301 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511d4b5c-a4c5-4f3b-8918-bd41a00104cc None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:45.974580 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511d4b5c-a4c5-4f3b-8918-bd41a00104cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:45.991446 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.auth.core [None req-511d4b5c-a4c5-4f3b-8918-bd41a00104cc None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:19:45.992641 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-511d4b5c-a4c5-4f3b-8918-bd41a00104cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:19:46.060117 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:19:46.064076 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 530/1057] 10.222.0.49 () {58 vars in 1023 bytes} [Wed Jun 15 10:19:45 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 15 10:19:46.065104 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e933da56-9481-42ac-884d-40164e56c675 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:19:46.086481 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511d4b5c-a4c5-4f3b-8918-bd41a00104cc None None] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:19:46.100821 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338959f6-f2ac-420f-b1ba-e6016a51283f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:19:46.101738 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338959f6-f2ac-420f-b1ba-e6016a51283f None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:46.102024 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338959f6-f2ac-420f-b1ba-e6016a51283f None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:46.102820 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338959f6-f2ac-420f-b1ba-e6016a51283f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:46.104410 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=100 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:19:46.110834 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-338959f6-f2ac-420f-b1ba-e6016a51283f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:19:46.112324 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-338959f6-f2ac-420f-b1ba-e6016a51283f None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:19:46.127358 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application [None req-338959f6-f2ac-420f-b1ba-e6016a51283f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 10:19:46.127358 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:19:46.127358 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:19:46.127358 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:19:46.127358 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:19:46.127358 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:19:46.127358 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:19:46.127358 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:19:46.127358 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:19:46.127358 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:19:46.127358 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:19:46.127358 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:19:46.127358 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 15 10:19:46.127358 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 15 10:19:46.127358 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 15 10:19:46.128269 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 15 10:19:46.128269 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:19:46.128269 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:19:46.128269 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:19:46.128269 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:19:46.128269 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:19:46.128269 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:19:46.128269 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:19:46.128269 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application with Lock( Jun 15 10:19:46.128269 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:19:46.128269 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:19:46.128269 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:19:46.128269 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:19:46.128269 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:19:46.128269 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:19:46.128269 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:19:46.128269 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:19:46.129174 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 15 10:19:46.129174 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 15 10:19:46.129174 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 15 10:19:46.129174 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 15 10:19:46.129174 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 15 10:19:46.129174 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 15 10:19:46.129174 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 10:19:46.129174 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Jun 15 10:19:46.129174 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 528/1058] 10.222.0.49 () {56 vars in 1111 bytes} [Wed Jun 15 10:19:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 10:19:46.134210 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338959f6-f2ac-420f-b1ba-e6016a51283f None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:19:46.196131 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511d4b5c-a4c5-4f3b-8918-bd41a00104cc None None] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:19:46.231019 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155e11e4-ce54-46e6-a9fd-1060de75a0d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:19:46.231834 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155e11e4-ce54-46e6-a9fd-1060de75a0d1 None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:46.232121 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155e11e4-ce54-46e6-a9fd-1060de75a0d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:46.232393 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155e11e4-ce54-46e6-a9fd-1060de75a0d1 None admin] PATH_INFO: `/v3/services/image` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:46.233026 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-155e11e4-ce54-46e6-a9fd-1060de75a0d1 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:19:46.234396 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-155e11e4-ce54-46e6-a9fd-1060de75a0d1 None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:19:46.238335 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338959f6-f2ac-420f-b1ba-e6016a51283f None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:19:46.246327 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application [None req-155e11e4-ce54-46e6-a9fd-1060de75a0d1 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jun 15 10:19:46.246327 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:19:46.246327 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 15 10:19:46.246327 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 15 10:19:46.246327 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 15 10:19:46.246327 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 15 10:19:46.246327 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 15 10:19:46.246327 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 10:19:46.246327 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 15 10:19:46.246327 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application Jun 15 10:19:46.246327 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 15 10:19:46.246327 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application Jun 15 10:19:46.246327 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:19:46.246327 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:19:46.246327 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:19:46.246327 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:19:46.246327 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:19:46.247380 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:19:46.247380 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:19:46.247380 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:19:46.247380 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:19:46.247380 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:19:46.247380 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:19:46.247380 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 15 10:19:46.247380 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 15 10:19:46.247380 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 15 10:19:46.247380 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 15 10:19:46.247380 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:19:46.247380 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:19:46.247380 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:19:46.247380 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:19:46.247380 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:19:46.247380 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:19:46.248241 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:19:46.248241 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application with Lock( Jun 15 10:19:46.248241 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:19:46.248241 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:19:46.248241 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:19:46.248241 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:19:46.248241 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:19:46.248241 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:19:46.248241 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:19:46.248241 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:19:46.248241 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 15 10:19:46.248241 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 10:19:46.248241 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 15 10:19:46.248241 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application Jun 15 10:19:46.248241 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 531/1059] 10.222.0.49 () {56 vars in 1105 bytes} [Wed Jun 15 10:19:46 2022] GET /identity/v3/services/image => generated 86 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 10:19:46.253222 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155e11e4-ce54-46e6-a9fd-1060de75a0d1 None admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:19:46.278664 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37418864-dd4c-4578-97c1-bd4fa8e7a419 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:19:46.279479 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37418864-dd4c-4578-97c1-bd4fa8e7a419 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:46.279701 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37418864-dd4c-4578-97c1-bd4fa8e7a419 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:46.279917 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37418864-dd4c-4578-97c1-bd4fa8e7a419 None admin] PATH_INFO: `/v3/projects` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:46.280707 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37418864-dd4c-4578-97c1-bd4fa8e7a419 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:19:46.281277 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37418864-dd4c-4578-97c1-bd4fa8e7a419 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:19:46.282431 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37418864-dd4c-4578-97c1-bd4fa8e7a419 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:19:46.295007 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 529/1060] 10.222.0.49 () {56 vars in 1148 bytes} [Wed Jun 15 10:19:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 10:19:46.300877 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37418864-dd4c-4578-97c1-bd4fa8e7a419 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:19:46.320802 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:19:46.349393 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=100 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:19:46.368915 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155e11e4-ce54-46e6-a9fd-1060de75a0d1 None admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:19:46.405043 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37418864-dd4c-4578-97c1-bd4fa8e7a419 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:19:46.406011 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8100f68-fd39-4a82-a8ec-784eacd5eb6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:19:46.407053 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8100f68-fd39-4a82-a8ec-784eacd5eb6b None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:46.407604 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8100f68-fd39-4a82-a8ec-784eacd5eb6b None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:46.407883 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8100f68-fd39-4a82-a8ec-784eacd5eb6b None admin] PATH_INFO: `/v3/services` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:46.408581 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8100f68-fd39-4a82-a8ec-784eacd5eb6b None admin] RBAC: Adding query filter params (name=image) {{(pid=80119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:19:46.408884 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8100f68-fd39-4a82-a8ec-784eacd5eb6b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:19:46.410248 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8100f68-fd39-4a82-a8ec-784eacd5eb6b None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:19:46.420080 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 532/1061] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Jun 15 10:19:46 2022] GET /identity/v3/services?name=image => generated 122 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 10:19:46.421020 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8100f68-fd39-4a82-a8ec-784eacd5eb6b None admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:19:46.445639 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2fbfaf-3579-4d76-ab12-2a16a9fd72d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:19:46.447163 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2fbfaf-3579-4d76-ab12-2a16a9fd72d9 None admin] REQUEST_METHOD: `PUT` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:46.447453 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2fbfaf-3579-4d76-ab12-2a16a9fd72d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:46.447733 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2fbfaf-3579-4d76-ab12-2a16a9fd72d9 None admin] PATH_INFO: `/v3/projects/559b3e713f5844b6ae37151d56dd419f/users/5ac54528370d4747b6e8777fba9d0c82/roles/d962cdb5fdb24466a560f5d7bcb1e034` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:46.450039 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af2fbfaf-3579-4d76-ab12-2a16a9fd72d9 None admin] RBAC: Authorizing `identity:create_grant(project_id=559b3e713f5844b6ae37151d56dd419f, user_id=5ac54528370d4747b6e8777fba9d0c82, role_id=d962cdb5fdb24466a560f5d7bcb1e034)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:19:46.451494 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af2fbfaf-3579-4d76-ab12-2a16a9fd72d9 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:19:46.469333 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 530/1062] 10.222.0.49 () {58 vars in 1439 bytes} [Wed Jun 15 10:19:46 2022] PUT /identity/v3/projects/559b3e713f5844b6ae37151d56dd419f/users/5ac54528370d4747b6e8777fba9d0c82/roles/d962cdb5fdb24466a560f5d7bcb1e034 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 10:19:46.541776 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8100f68-fd39-4a82-a8ec-784eacd5eb6b None admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:19:46.578997 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b40b1c3-659d-474b-b7e6-8feeab74f657 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:19:46.579712 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b40b1c3-659d-474b-b7e6-8feeab74f657 None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:46.579999 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b40b1c3-659d-474b-b7e6-8feeab74f657 None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:46.580276 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b40b1c3-659d-474b-b7e6-8feeab74f657 None admin] PATH_INFO: `/v3/services` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:46.581050 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b40b1c3-659d-474b-b7e6-8feeab74f657 None admin] RBAC: Adding query filter params (type=image) {{(pid=80119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:19:46.581363 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b40b1c3-659d-474b-b7e6-8feeab74f657 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:19:46.582766 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b40b1c3-659d-474b-b7e6-8feeab74f657 None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:19:46.593108 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 533/1063] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Jun 15 10:19:46 2022] GET /identity/v3/services?type=image => generated 350 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:19:46.594585 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2fbfaf-3579-4d76-ab12-2a16a9fd72d9 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:19:46.682544 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:19:46.705530 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2fbfaf-3579-4d76-ab12-2a16a9fd72d9 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:19:46.740077 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b13216b-08f1-4522-b20e-8b8b6462a44b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:19:46.740736 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b13216b-08f1-4522-b20e-8b8b6462a44b None admin] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:46.741018 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b13216b-08f1-4522-b20e-8b8b6462a44b None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:46.747651 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b13216b-08f1-4522-b20e-8b8b6462a44b None admin] PATH_INFO: `/v3/endpoints` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:46.750078 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b13216b-08f1-4522-b20e-8b8b6462a44b None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:19:46.752564 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b13216b-08f1-4522-b20e-8b8b6462a44b None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:19:46.780065 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 531/1064] 10.222.0.49 () {60 vars in 1144 bytes} [Wed Jun 15 10:19:46 2022] POST /identity/v3/endpoints => generated 327 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 10:19:47.123003 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c2cc91-2d69-4737-8a63-381c4ad0b812 None None] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:47.123422 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c2cc91-2d69-4737-8a63-381c4ad0b812 None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:47.123624 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c2cc91-2d69-4737-8a63-381c4ad0b812 None None] PATH_INFO: `/` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:47.125132 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 534/1065] 10.222.0.49 () {54 vars in 926 bytes} [Wed Jun 15 10:19:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:19:47.131251 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aeb174a-4f4b-497b-aee2-e6039c48b744 None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:47.131539 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aeb174a-4f4b-497b-aee2-e6039c48b744 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:47.131816 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aeb174a-4f4b-497b-aee2-e6039c48b744 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:47.149810 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-2aeb174a-4f4b-497b-aee2-e6039c48b744 None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:19:47.151013 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-2aeb174a-4f4b-497b-aee2-e6039c48b744 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:19:47.232715 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 532/1066] 10.222.0.49 () {58 vars in 1023 bytes} [Wed Jun 15 10:19:47 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 10:19:47.256372 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c2cc91-2d69-4737-8a63-381c4ad0b812 None None] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:19:47.269650 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=100 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:19:47.365202 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c2cc91-2d69-4737-8a63-381c4ad0b812 None None] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:19:47.400629 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6469ab2d-548c-4faf-9941-4498ad44fcea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:19:47.401352 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6469ab2d-548c-4faf-9941-4498ad44fcea None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:47.401640 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6469ab2d-548c-4faf-9941-4498ad44fcea None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:47.401915 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6469ab2d-548c-4faf-9941-4498ad44fcea None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:47.412554 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6469ab2d-548c-4faf-9941-4498ad44fcea None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 10:19:47.420884 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application [None req-6469ab2d-548c-4faf-9941-4498ad44fcea None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 10:19:47.420884 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:19:47.420884 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:19:47.420884 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:19:47.420884 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:19:47.420884 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:19:47.420884 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:19:47.420884 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:19:47.420884 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:19:47.420884 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:19:47.420884 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:19:47.420884 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:19:47.420884 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 10:19:47.420884 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 10:19:47.420884 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 10:19:47.420884 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 10:19:47.421798 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 10:19:47.421798 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 10:19:47.421798 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 10:19:47.421798 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 10:19:47.421798 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:19:47.421798 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:19:47.421798 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:19:47.421798 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:19:47.421798 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:19:47.421798 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:19:47.421798 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:19:47.421798 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application with Lock( Jun 15 10:19:47.421798 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:19:47.421798 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:19:47.421798 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:19:47.421798 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:19:47.428115 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:19:47.428115 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:19:47.428115 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:19:47.428115 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:19:47.428115 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 10:19:47.428115 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 10:19:47.428115 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 10:19:47.428115 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 10:19:47.428115 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 10:19:47.428115 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 10:19:47.428115 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 10:19:47.428115 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application Jun 15 10:19:47.428115 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 535/1067] 10.222.0.49 () {56 vars in 1096 bytes} [Wed Jun 15 10:19:47 2022] GET /identity/v3/roles/admin => generated 83 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:19:47.430888 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aeb174a-4f4b-497b-aee2-e6039c48b744 None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:19:47.489450 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:19:47.533245 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aeb174a-4f4b-497b-aee2-e6039c48b744 None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:19:47.569024 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371fa107-0f22-42bd-9829-c498c3e5b5f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:19:47.569696 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371fa107-0f22-42bd-9829-c498c3e5b5f4 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:47.569976 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371fa107-0f22-42bd-9829-c498c3e5b5f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:47.570246 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371fa107-0f22-42bd-9829-c498c3e5b5f4 None admin] PATH_INFO: `/v3/roles` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:47.571022 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-371fa107-0f22-42bd-9829-c498c3e5b5f4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:19:47.571356 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-371fa107-0f22-42bd-9829-c498c3e5b5f4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:19:47.572722 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-371fa107-0f22-42bd-9829-c498c3e5b5f4 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:19:47.585811 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 533/1068] 10.222.0.49 () {56 vars in 1099 bytes} [Wed Jun 15 10:19:47 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:19:47.587583 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6469ab2d-548c-4faf-9941-4498ad44fcea None admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:19:47.657816 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=100 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:19:47.693186 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6469ab2d-548c-4faf-9941-4498ad44fcea None admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:19:47.726113 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45288e02-2c01-4015-b09b-bd1be5958442 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:19:47.726982 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45288e02-2c01-4015-b09b-bd1be5958442 None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:47.727821 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45288e02-2c01-4015-b09b-bd1be5958442 None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:47.728102 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45288e02-2c01-4015-b09b-bd1be5958442 None admin] PATH_INFO: `/v3/users/6fb33fc6db7a459592f21e15977818d0` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:47.741150 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45288e02-2c01-4015-b09b-bd1be5958442 None admin] RBAC: Authorizing `identity:get_user(user_id=6fb33fc6db7a459592f21e15977818d0)` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:19:47.742607 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45288e02-2c01-4015-b09b-bd1be5958442 None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:19:47.745053 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 536/1069] 10.222.0.49 () {56 vars in 1177 bytes} [Wed Jun 15 10:19:47 2022] GET /identity/v3/users/6fb33fc6db7a459592f21e15977818d0 => generated 285 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:19:47.750499 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371fa107-0f22-42bd-9829-c498c3e5b5f4 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:19:47.858320 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371fa107-0f22-42bd-9829-c498c3e5b5f4 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:19:47.880852 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:19:47.894450 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e62a8a-623a-48a8-83a4-e1a1a1ad53ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:19:47.895123 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e62a8a-623a-48a8-83a4-e1a1a1ad53ca None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:47.895412 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e62a8a-623a-48a8-83a4-e1a1a1ad53ca None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:47.895688 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e62a8a-623a-48a8-83a4-e1a1a1ad53ca None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:47.963144 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e62a8a-623a-48a8-83a4-e1a1a1ad53ca None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:19:47.964499 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e62a8a-623a-48a8-83a4-e1a1a1ad53ca None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:19:48.066503 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 534/1070] 10.222.0.49 () {58 vars in 1282 bytes} [Wed Jun 15 10:19:47 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 15 10:19:48.068082 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45288e02-2c01-4015-b09b-bd1be5958442 None admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:19:48.168613 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45288e02-2c01-4015-b09b-bd1be5958442 None admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:19:48.194528 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=100 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:19:48.204247 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91406e93-54aa-4f0f-8e2a-4ccd8bbc2418 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:19:48.205114 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91406e93-54aa-4f0f-8e2a-4ccd8bbc2418 None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:48.205398 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91406e93-54aa-4f0f-8e2a-4ccd8bbc2418 None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:48.205675 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91406e93-54aa-4f0f-8e2a-4ccd8bbc2418 None admin] PATH_INFO: `/v3/projects/c0fb512b3149496cb3113317113a0fe1` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:48.206854 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91406e93-54aa-4f0f-8e2a-4ccd8bbc2418 None admin] RBAC: Authorizing `identity:get_project(project_id=c0fb512b3149496cb3113317113a0fe1)` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:19:48.208321 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91406e93-54aa-4f0f-8e2a-4ccd8bbc2418 None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:19:48.215017 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 537/1071] 10.222.0.49 () {56 vars in 1186 bytes} [Wed Jun 15 10:19:48 2022] GET /identity/v3/projects/c0fb512b3149496cb3113317113a0fe1 => generated 311 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:19:48.216550 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e62a8a-623a-48a8-83a4-e1a1a1ad53ca None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:19:48.241236 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:19:48.319473 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e62a8a-623a-48a8-83a4-e1a1a1ad53ca None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:19:48.353599 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baaa255d-2e4e-4df7-9570-7ce2078a3644 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:19:48.354262 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baaa255d-2e4e-4df7-9570-7ce2078a3644 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:48.354547 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baaa255d-2e4e-4df7-9570-7ce2078a3644 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:48.354822 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baaa255d-2e4e-4df7-9570-7ce2078a3644 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:48.355684 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baaa255d-2e4e-4df7-9570-7ce2078a3644 None admin] RBAC: Adding query filter params (role.id=8051174e7a7947d5b991ebd793c2e543, scope.project.id=c0fb512b3149496cb3113317113a0fe1, user.id=6fb33fc6db7a459592f21e15977818d0) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:19:48.355997 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baaa255d-2e4e-4df7-9570-7ce2078a3644 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:19:48.357426 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baaa255d-2e4e-4df7-9570-7ce2078a3644 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:19:48.371358 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 535/1072] 10.222.0.49 () {56 vars in 1374 bytes} [Wed Jun 15 10:19:48 2022] GET /identity/v3/role_assignments?user.id=6fb33fc6db7a459592f21e15977818d0&scope.project.id=c0fb512b3149496cb3113317113a0fe1&role.id=8051174e7a7947d5b991ebd793c2e543 => generated 259 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:19:48.631591 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abf0a09-e1f4-4281-8c8c-239f3426bf30 None None] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:48.632143 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abf0a09-e1f4-4281-8c8c-239f3426bf30 None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:48.632143 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abf0a09-e1f4-4281-8c8c-239f3426bf30 None None] PATH_INFO: `/` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:48.633692 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 538/1073] 10.222.0.49 () {54 vars in 926 bytes} [Wed Jun 15 10:19:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:19:48.638597 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f45f78c-6f9e-441e-af46-d87014d513cf None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:48.638971 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f45f78c-6f9e-441e-af46-d87014d513cf None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:48.639174 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f45f78c-6f9e-441e-af46-d87014d513cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:48.656215 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-2f45f78c-6f9e-441e-af46-d87014d513cf None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:19:48.657407 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-2f45f78c-6f9e-441e-af46-d87014d513cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:19:48.725712 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 536/1074] 10.222.0.49 () {58 vars in 1023 bytes} [Wed Jun 15 10:19:48 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 10:19:48.749174 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abf0a09-e1f4-4281-8c8c-239f3426bf30 None None] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:19:48.860231 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abf0a09-e1f4-4281-8c8c-239f3426bf30 None None] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:19:48.898639 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4fdfa0f-191e-4c83-8c80-473de1e39fbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:19:48.899957 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fdfa0f-191e-4c83-8c80-473de1e39fbc None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:48.900239 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fdfa0f-191e-4c83-8c80-473de1e39fbc None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:48.900512 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fdfa0f-191e-4c83-8c80-473de1e39fbc None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:48.911386 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4fdfa0f-191e-4c83-8c80-473de1e39fbc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 10:19:48.919506 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application [None req-a4fdfa0f-191e-4c83-8c80-473de1e39fbc None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 15 10:19:48.919506 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:19:48.919506 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:19:48.919506 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:19:48.919506 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:19:48.919506 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:19:48.919506 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:19:48.919506 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:19:48.919506 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:19:48.919506 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:19:48.919506 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:19:48.919506 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:19:48.919506 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 10:19:48.919506 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 10:19:48.919506 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 10:19:48.919506 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 10:19:48.920423 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 10:19:48.920423 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 10:19:48.920423 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 10:19:48.920423 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 10:19:48.920423 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:19:48.920423 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:19:48.920423 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:19:48.920423 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:19:48.920423 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:19:48.920423 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:19:48.920423 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:19:48.920423 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application with Lock( Jun 15 10:19:48.920423 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:19:48.920423 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:19:48.920423 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:19:48.920423 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:19:48.921259 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:19:48.921259 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:19:48.921259 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:19:48.921259 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:19:48.921259 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 10:19:48.921259 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 10:19:48.921259 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 10:19:48.921259 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 10:19:48.921259 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 10:19:48.921259 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 10:19:48.921259 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 15 10:19:48.921259 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application Jun 15 10:19:48.923434 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 539/1075] 10.222.0.49 () {56 vars in 1120 bytes} [Wed Jun 15 10:19:48 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:19:48.926811 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f45f78c-6f9e-441e-af46-d87014d513cf None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:19:48.933922 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31df7016-d0bc-47af-897e-bbddee387cec None None] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:48.934216 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31df7016-d0bc-47af-897e-bbddee387cec None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:48.934503 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31df7016-d0bc-47af-897e-bbddee387cec None None] PATH_INFO: `/` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:48.936001 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 540/1076] 10.222.0.49 () {54 vars in 926 bytes} [Wed Jun 15 10:19:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 15 10:19:48.940528 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637001fb-d80b-4500-9292-bd2c0776b70d None None] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:48.940823 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637001fb-d80b-4500-9292-bd2c0776b70d None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:48.941106 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637001fb-d80b-4500-9292-bd2c0776b70d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:48.945121 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:19:48.958484 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.auth.core [None req-637001fb-d80b-4500-9292-bd2c0776b70d None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:19:48.959709 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-637001fb-d80b-4500-9292-bd2c0776b70d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:19:49.011404 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=100 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:19:49.030743 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 541/1077] 10.222.0.49 () {58 vars in 1023 bytes} [Wed Jun 15 10:19:48 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 15 10:19:49.035792 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f45f78c-6f9e-441e-af46-d87014d513cf None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:19:49.053026 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637001fb-d80b-4500-9292-bd2c0776b70d None None] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:19:49.072317 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131a0b70-a6db-4672-b39d-8d09692d255e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:19:49.073281 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131a0b70-a6db-4672-b39d-8d09692d255e None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:49.073569 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131a0b70-a6db-4672-b39d-8d09692d255e None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:49.074012 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131a0b70-a6db-4672-b39d-8d09692d255e None admin] PATH_INFO: `/v3/roles` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:49.075063 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131a0b70-a6db-4672-b39d-8d09692d255e None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:19:49.075409 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131a0b70-a6db-4672-b39d-8d09692d255e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:19:49.076790 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131a0b70-a6db-4672-b39d-8d09692d255e None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:19:49.090376 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 537/1078] 10.222.0.49 () {56 vars in 1115 bytes} [Wed Jun 15 10:19:48 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 10:19:49.091685 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131a0b70-a6db-4672-b39d-8d09692d255e None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:19:49.163726 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637001fb-d80b-4500-9292-bd2c0776b70d None None] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:19:49.176871 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:19:49.201220 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1578b551-cdeb-4bf5-85ef-8a20dc045c0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:19:49.202110 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1578b551-cdeb-4bf5-85ef-8a20dc045c0f None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:49.202392 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1578b551-cdeb-4bf5-85ef-8a20dc045c0f None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:49.202664 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1578b551-cdeb-4bf5-85ef-8a20dc045c0f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:49.202823 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131a0b70-a6db-4672-b39d-8d09692d255e None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:19:49.239261 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afed6042-4a80-41ce-88ba-5aa9c574cb48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:19:49.240058 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afed6042-4a80-41ce-88ba-5aa9c574cb48 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:49.240348 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afed6042-4a80-41ce-88ba-5aa9c574cb48 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:49.240624 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afed6042-4a80-41ce-88ba-5aa9c574cb48 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:49.251172 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afed6042-4a80-41ce-88ba-5aa9c574cb48 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:19:49.252714 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afed6042-4a80-41ce-88ba-5aa9c574cb48 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:19:49.259936 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=100 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:19:49.270391 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application [None req-afed6042-4a80-41ce-88ba-5aa9c574cb48 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 10:19:49.270391 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:19:49.270391 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:19:49.270391 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:19:49.270391 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:19:49.270391 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:19:49.270391 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:19:49.270391 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:19:49.270391 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:19:49.270391 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:19:49.270391 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:19:49.270391 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:19:49.270391 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 10:19:49.270391 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 10:19:49.270391 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 10:19:49.271323 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 10:19:49.271323 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:19:49.271323 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:19:49.271323 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:19:49.271323 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:19:49.271323 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:19:49.271323 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:19:49.271323 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:19:49.271323 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application with Lock( Jun 15 10:19:49.271323 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:19:49.271323 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:19:49.271323 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:19:49.271323 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:19:49.271323 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:19:49.271323 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:19:49.271323 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:19:49.271323 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:19:49.272573 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 10:19:49.272573 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 10:19:49.272573 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 10:19:49.272573 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Jun 15 10:19:49.272573 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 538/1079] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Jun 15 10:19:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 10:19:49.277810 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afed6042-4a80-41ce-88ba-5aa9c574cb48 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:19:49.280128 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1578b551-cdeb-4bf5-85ef-8a20dc045c0f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:19:49.281623 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1578b551-cdeb-4bf5-85ef-8a20dc045c0f None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:19:49.385779 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afed6042-4a80-41ce-88ba-5aa9c574cb48 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:19:49.392187 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 542/1080] 10.222.0.49 () {58 vars in 1282 bytes} [Wed Jun 15 10:19:49 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 15 10:19:49.393349 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1578b551-cdeb-4bf5-85ef-8a20dc045c0f None admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:19:49.415045 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:19:49.422941 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d138e65-bfa6-4306-bc96-2b63bbd84324 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:19:49.423598 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d138e65-bfa6-4306-bc96-2b63bbd84324 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:49.424124 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d138e65-bfa6-4306-bc96-2b63bbd84324 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:49.424708 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d138e65-bfa6-4306-bc96-2b63bbd84324 None admin] PATH_INFO: `/v3/domains` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:49.425145 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d138e65-bfa6-4306-bc96-2b63bbd84324 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:19:49.425451 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d138e65-bfa6-4306-bc96-2b63bbd84324 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:19:49.426861 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d138e65-bfa6-4306-bc96-2b63bbd84324 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:19:49.440218 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 539/1081] 10.222.0.49 () {56 vars in 1109 bytes} [Wed Jun 15 10:19:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 10:19:49.446710 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d138e65-bfa6-4306-bc96-2b63bbd84324 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:19:49.485427 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=100 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:19:49.503215 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1578b551-cdeb-4bf5-85ef-8a20dc045c0f None admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:19:49.540664 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9220827c-f432-471e-b3d3-3b66835a3238 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:19:49.541463 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9220827c-f432-471e-b3d3-3b66835a3238 None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:49.542123 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9220827c-f432-471e-b3d3-3b66835a3238 None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:49.542408 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9220827c-f432-471e-b3d3-3b66835a3238 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:49.553093 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9220827c-f432-471e-b3d3-3b66835a3238 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:19:49.554606 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9220827c-f432-471e-b3d3-3b66835a3238 None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:19:49.562317 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d138e65-bfa6-4306-bc96-2b63bbd84324 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:19:49.574975 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application [None req-9220827c-f432-471e-b3d3-3b66835a3238 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 10:19:49.574975 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:19:49.574975 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:19:49.574975 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:19:49.574975 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:19:49.574975 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:19:49.574975 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:19:49.574975 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:19:49.574975 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:19:49.574975 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:19:49.574975 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:19:49.574975 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:19:49.574975 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 10:19:49.574975 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 10:19:49.574975 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 10:19:49.576034 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 10:19:49.576034 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:19:49.576034 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:19:49.576034 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:19:49.576034 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:19:49.576034 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:19:49.576034 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:19:49.576034 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:19:49.576034 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application with Lock( Jun 15 10:19:49.576034 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:19:49.576034 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:19:49.576034 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:19:49.576034 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:19:49.576034 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:19:49.576034 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:19:49.576034 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:19:49.576034 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:19:49.577595 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 10:19:49.577595 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 10:19:49.577595 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 10:19:49.577595 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application Jun 15 10:19:49.577595 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 543/1082] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Jun 15 10:19:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 10:19:49.581918 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9220827c-f432-471e-b3d3-3b66835a3238 None admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:19:49.601714 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02324a77-ba4d-4b0d-8479-ab20fa0046cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:19:49.602545 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02324a77-ba4d-4b0d-8479-ab20fa0046cf None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:49.602831 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02324a77-ba4d-4b0d-8479-ab20fa0046cf None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:49.603565 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02324a77-ba4d-4b0d-8479-ab20fa0046cf None admin] PATH_INFO: `/v3/users/nova` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:49.612969 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02324a77-ba4d-4b0d-8479-ab20fa0046cf None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:19:49.614539 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02324a77-ba4d-4b0d-8479-ab20fa0046cf None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:19:49.633024 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application [None req-02324a77-ba4d-4b0d-8479-ab20fa0046cf None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 15 10:19:49.633024 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:19:49.633024 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:19:49.633024 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:19:49.633024 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:19:49.633024 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:19:49.633024 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:19:49.633024 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:19:49.633024 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:19:49.633024 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:19:49.633024 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:19:49.633024 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:19:49.633024 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 15 10:19:49.633024 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 15 10:19:49.633024 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 15 10:19:49.633024 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 15 10:19:49.634572 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:19:49.634572 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:19:49.634572 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 15 10:19:49.634572 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 10:19:49.634572 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 15 10:19:49.634572 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 10:19:49.634572 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:19:49.634572 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:19:49.634572 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:19:49.634572 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:19:49.634572 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:19:49.634572 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application with Lock( Jun 15 10:19:49.634572 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:19:49.634572 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:19:49.634572 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:19:49.634572 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:19:49.634572 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:19:49.634572 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:19:49.635570 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:19:49.635570 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:19:49.635570 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 15 10:19:49.635570 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 15 10:19:49.635570 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 15 10:19:49.635570 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 15 10:19:49.635570 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 15 10:19:49.635570 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 15 10:19:49.635570 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 15 10:19:49.635570 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Jun 15 10:19:49.636868 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 540/1083] 10.222.0.49 () {56 vars in 1093 bytes} [Wed Jun 15 10:19:49 2022] GET /identity/v3/users/nova => generated 82 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 10:19:49.640884 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02324a77-ba4d-4b0d-8479-ab20fa0046cf None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:19:49.689861 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9220827c-f432-471e-b3d3-3b66835a3238 None admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:19:49.691342 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:19:49.725843 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1727fb43-611e-41d5-bdd5-99f254f6e5ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:19:49.726789 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1727fb43-611e-41d5-bdd5-99f254f6e5ef None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:49.727786 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1727fb43-611e-41d5-bdd5-99f254f6e5ef None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:49.728215 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1727fb43-611e-41d5-bdd5-99f254f6e5ef None admin] PATH_INFO: `/v3/domains` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:49.729081 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1727fb43-611e-41d5-bdd5-99f254f6e5ef None admin] RBAC: Adding query filter params (name=Default) {{(pid=80119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:19:49.729560 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1727fb43-611e-41d5-bdd5-99f254f6e5ef None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:19:49.731106 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1727fb43-611e-41d5-bdd5-99f254f6e5ef None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:19:49.733956 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=100 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:19:49.744323 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 544/1084] 10.222.0.49 () {56 vars in 1109 bytes} [Wed Jun 15 10:19:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 10:19:49.745271 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02324a77-ba4d-4b0d-8479-ab20fa0046cf None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:19:49.797076 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9fdada-00ec-4fb1-af39-df3bd0e6099b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:19:49.797796 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9fdada-00ec-4fb1-af39-df3bd0e6099b None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:49.798085 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9fdada-00ec-4fb1-af39-df3bd0e6099b None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:49.798363 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9fdada-00ec-4fb1-af39-df3bd0e6099b None admin] PATH_INFO: `/v3/users` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:49.799512 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9fdada-00ec-4fb1-af39-df3bd0e6099b None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:19:49.799826 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9fdada-00ec-4fb1-af39-df3bd0e6099b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:19:49.801316 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9fdada-00ec-4fb1-af39-df3bd0e6099b None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:19:49.827816 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 541/1085] 10.222.0.49 () {56 vars in 1133 bytes} [Wed Jun 15 10:19:49 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 10:19:49.829214 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1727fb43-611e-41d5-bdd5-99f254f6e5ef None admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:19:49.937403 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1727fb43-611e-41d5-bdd5-99f254f6e5ef None admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:19:49.973576 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be01aab-41f5-4876-84e9-8153f1a6deb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:19:49.974172 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be01aab-41f5-4876-84e9-8153f1a6deb5 None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:49.974466 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be01aab-41f5-4876-84e9-8153f1a6deb5 None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:49.974745 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be01aab-41f5-4876-84e9-8153f1a6deb5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:50.040362 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=100 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:19:50.048685 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be01aab-41f5-4876-84e9-8153f1a6deb5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:19:50.050431 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be01aab-41f5-4876-84e9-8153f1a6deb5 None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:19:50.147844 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 545/1086] 10.222.0.49 () {58 vars in 1282 bytes} [Wed Jun 15 10:19:49 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 15 10:19:50.154015 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9fdada-00ec-4fb1-af39-df3bd0e6099b None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:19:50.250355 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9fdada-00ec-4fb1-af39-df3bd0e6099b None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:19:50.259270 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:19:50.283630 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d792d9e5-0931-44c9-8301-9b28eb11bb5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:19:50.284420 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d792d9e5-0931-44c9-8301-9b28eb11bb5e None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:50.284700 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d792d9e5-0931-44c9-8301-9b28eb11bb5e None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:50.284971 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d792d9e5-0931-44c9-8301-9b28eb11bb5e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:50.295440 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d792d9e5-0931-44c9-8301-9b28eb11bb5e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:19:50.296871 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d792d9e5-0931-44c9-8301-9b28eb11bb5e None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:19:50.314035 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application [None req-d792d9e5-0931-44c9-8301-9b28eb11bb5e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 10:19:50.314035 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:19:50.314035 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:19:50.314035 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:19:50.314035 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:19:50.314035 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:19:50.314035 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:19:50.314035 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:19:50.314035 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:19:50.314035 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:19:50.314035 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:19:50.314035 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:19:50.314035 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 10:19:50.314035 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 10:19:50.314035 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 10:19:50.315164 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 10:19:50.315164 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:19:50.315164 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:19:50.315164 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:19:50.315164 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:19:50.315164 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:19:50.315164 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:19:50.315164 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:19:50.315164 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application with Lock( Jun 15 10:19:50.315164 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:19:50.315164 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:19:50.315164 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:19:50.315164 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:19:50.315164 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:19:50.315164 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:19:50.315164 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:19:50.315164 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:19:50.320819 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 10:19:50.320819 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 10:19:50.320819 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 10:19:50.320819 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Jun 15 10:19:50.320819 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 542/1087] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Jun 15 10:19:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:19:50.322232 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be01aab-41f5-4876-84e9-8153f1a6deb5 None admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:19:50.415884 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=100 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:19:50.422410 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be01aab-41f5-4876-84e9-8153f1a6deb5 None admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:19:50.457557 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ccdd07-de7b-45bb-9bca-34bda20e7b15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:19:50.458228 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ccdd07-de7b-45bb-9bca-34bda20e7b15 None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:50.458512 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ccdd07-de7b-45bb-9bca-34bda20e7b15 None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:50.458788 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ccdd07-de7b-45bb-9bca-34bda20e7b15 None admin] PATH_INFO: `/v3/domains` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:50.459540 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ccdd07-de7b-45bb-9bca-34bda20e7b15 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:19:50.459848 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ccdd07-de7b-45bb-9bca-34bda20e7b15 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:19:50.461217 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ccdd07-de7b-45bb-9bca-34bda20e7b15 None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:19:50.465329 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6032b8-a51b-4538-982a-c652f63e71d8 None None] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:50.465621 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6032b8-a51b-4538-982a-c652f63e71d8 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:50.465899 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6032b8-a51b-4538-982a-c652f63e71d8 None None] PATH_INFO: `/` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:50.467318 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 543/1088] 10.222.0.49 () {54 vars in 926 bytes} [Wed Jun 15 10:19:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 15 10:19:50.472178 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c85def9-3ba7-43ba-b239-580c6588fa1e None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:50.472469 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c85def9-3ba7-43ba-b239-580c6588fa1e None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:50.472748 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c85def9-3ba7-43ba-b239-580c6588fa1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:50.473185 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 546/1089] 10.222.0.49 () {56 vars in 1109 bytes} [Wed Jun 15 10:19:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:19:50.478596 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ccdd07-de7b-45bb-9bca-34bda20e7b15 None admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:19:50.489297 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-1c85def9-3ba7-43ba-b239-580c6588fa1e None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:19:50.490499 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-1c85def9-3ba7-43ba-b239-580c6588fa1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:19:50.557180 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 544/1090] 10.222.0.49 () {58 vars in 1023 bytes} [Wed Jun 15 10:19:50 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 15 10:19:50.578237 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ccdd07-de7b-45bb-9bca-34bda20e7b15 None admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:19:50.579506 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c85def9-3ba7-43ba-b239-580c6588fa1e None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:19:50.612637 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4cedf2f-3ae2-4e94-abaa-d238f779f8b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:19:50.613428 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cedf2f-3ae2-4e94-abaa-d238f779f8b3 None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:50.613710 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cedf2f-3ae2-4e94-abaa-d238f779f8b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:50.613981 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cedf2f-3ae2-4e94-abaa-d238f779f8b3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:50.620926 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4cedf2f-3ae2-4e94-abaa-d238f779f8b3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:19:50.622361 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4cedf2f-3ae2-4e94-abaa-d238f779f8b3 None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:19:50.636671 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application [None req-b4cedf2f-3ae2-4e94-abaa-d238f779f8b3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 10:19:50.636671 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:19:50.636671 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:19:50.636671 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:19:50.636671 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:19:50.636671 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:19:50.636671 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:19:50.636671 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:19:50.636671 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:19:50.636671 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:19:50.636671 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:19:50.636671 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:19:50.636671 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 15 10:19:50.636671 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 15 10:19:50.636671 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 15 10:19:50.637676 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 15 10:19:50.637676 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:19:50.637676 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:19:50.637676 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:19:50.637676 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:19:50.637676 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:19:50.637676 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:19:50.637676 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:19:50.637676 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application with Lock( Jun 15 10:19:50.637676 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:19:50.637676 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:19:50.637676 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:19:50.637676 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:19:50.637676 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:19:50.637676 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:19:50.637676 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:19:50.637676 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:19:50.639858 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 15 10:19:50.639858 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 15 10:19:50.639858 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 15 10:19:50.639858 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 15 10:19:50.639858 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 15 10:19:50.639858 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 15 10:19:50.639858 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 10:19:50.639858 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application Jun 15 10:19:50.639858 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 547/1091] 10.222.0.49 () {56 vars in 1111 bytes} [Wed Jun 15 10:19:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 10:19:50.643495 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4cedf2f-3ae2-4e94-abaa-d238f779f8b3 None admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:19:50.661336 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=100 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:19:50.670820 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:19:50.680769 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c85def9-3ba7-43ba-b239-580c6588fa1e None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:19:50.714809 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e756b17f-a633-4b6d-b274-0bc7984844bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:19:50.715629 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e756b17f-a633-4b6d-b274-0bc7984844bc None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:50.715913 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e756b17f-a633-4b6d-b274-0bc7984844bc None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:50.716190 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e756b17f-a633-4b6d-b274-0bc7984844bc None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:50.726066 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e756b17f-a633-4b6d-b274-0bc7984844bc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 10:19:50.733711 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application [None req-e756b17f-a633-4b6d-b274-0bc7984844bc None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 10:19:50.733711 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:19:50.733711 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:19:50.733711 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:19:50.733711 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:19:50.733711 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:19:50.733711 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:19:50.733711 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:19:50.733711 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:19:50.733711 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:19:50.733711 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:19:50.733711 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:19:50.733711 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 10:19:50.733711 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 10:19:50.733711 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 10:19:50.733711 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 10:19:50.734752 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 10:19:50.734752 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 10:19:50.734752 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 10:19:50.734752 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 10:19:50.734752 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:19:50.734752 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:19:50.734752 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:19:50.734752 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:19:50.734752 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:19:50.734752 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:19:50.734752 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:19:50.734752 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application with Lock( Jun 15 10:19:50.734752 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:19:50.734752 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:19:50.734752 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:19:50.734752 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:19:50.735684 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:19:50.735684 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:19:50.735684 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:19:50.735684 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:19:50.735684 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 10:19:50.735684 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 10:19:50.735684 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 10:19:50.735684 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 10:19:50.735684 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 10:19:50.735684 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 10:19:50.735684 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 10:19:50.735684 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Jun 15 10:19:50.735684 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 545/1092] 10.222.0.49 () {56 vars in 1096 bytes} [Wed Jun 15 10:19:50 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 10:19:50.740874 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e756b17f-a633-4b6d-b274-0bc7984844bc None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:19:50.744290 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4cedf2f-3ae2-4e94-abaa-d238f779f8b3 None admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:19:50.778786 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45216441-09cd-482f-985c-addf8240936e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:19:50.779533 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45216441-09cd-482f-985c-addf8240936e None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:50.779816 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45216441-09cd-482f-985c-addf8240936e None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:50.780090 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45216441-09cd-482f-985c-addf8240936e None admin] PATH_INFO: `/v3/projects` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:50.780861 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45216441-09cd-482f-985c-addf8240936e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:19:50.781165 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45216441-09cd-482f-985c-addf8240936e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:19:50.782881 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45216441-09cd-482f-985c-addf8240936e None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:19:50.794816 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 548/1093] 10.222.0.49 () {56 vars in 1148 bytes} [Wed Jun 15 10:19:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 10:19:50.800939 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45216441-09cd-482f-985c-addf8240936e None admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:19:50.846373 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e756b17f-a633-4b6d-b274-0bc7984844bc None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:19:50.875681 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=100 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:19:50.879688 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8ed878-bba2-499a-acbb-33c60ec493ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:19:50.880359 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8ed878-bba2-499a-acbb-33c60ec493ff None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:50.880644 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8ed878-bba2-499a-acbb-33c60ec493ff None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:50.880922 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8ed878-bba2-499a-acbb-33c60ec493ff None admin] PATH_INFO: `/v3/roles` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:50.881665 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd8ed878-bba2-499a-acbb-33c60ec493ff None admin] RBAC: Adding query filter params (name=admin) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:19:50.881971 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd8ed878-bba2-499a-acbb-33c60ec493ff None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:19:50.883902 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd8ed878-bba2-499a-acbb-33c60ec493ff None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:19:50.891917 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 546/1094] 10.222.0.49 () {56 vars in 1099 bytes} [Wed Jun 15 10:19:50 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 10:19:50.897473 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8ed878-bba2-499a-acbb-33c60ec493ff None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:19:50.899441 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45216441-09cd-482f-985c-addf8240936e None admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:19:50.911670 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:19:50.933735 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e76dd3-44f3-40c4-bfd8-b39c600bb3a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:19:50.934388 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e76dd3-44f3-40c4-bfd8-b39c600bb3a9 None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:50.934669 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e76dd3-44f3-40c4-bfd8-b39c600bb3a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:50.934945 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e76dd3-44f3-40c4-bfd8-b39c600bb3a9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:50.935804 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e76dd3-44f3-40c4-bfd8-b39c600bb3a9 None admin] RBAC: Adding query filter params (role.id=d962cdb5fdb24466a560f5d7bcb1e034, scope.project.id=559b3e713f5844b6ae37151d56dd419f, user.id=5ac54528370d4747b6e8777fba9d0c82) {{(pid=80119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:19:50.936118 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e76dd3-44f3-40c4-bfd8-b39c600bb3a9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:19:50.937573 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e76dd3-44f3-40c4-bfd8-b39c600bb3a9 None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:19:50.943463 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 549/1095] 10.222.0.49 () {56 vars in 1374 bytes} [Wed Jun 15 10:19:50 2022] GET /identity/v3/role_assignments?user.id=5ac54528370d4747b6e8777fba9d0c82&scope.project.id=559b3e713f5844b6ae37151d56dd419f&role.id=d962cdb5fdb24466a560f5d7bcb1e034 => generated 608 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 10:19:51.002668 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8ed878-bba2-499a-acbb-33c60ec493ff None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:19:51.038729 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beea3e66-e94b-4a92-92d0-f34fd70c7906 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:19:51.039692 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beea3e66-e94b-4a92-92d0-f34fd70c7906 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:51.040169 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beea3e66-e94b-4a92-92d0-f34fd70c7906 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:51.040455 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beea3e66-e94b-4a92-92d0-f34fd70c7906 None admin] PATH_INFO: `/v3/users/6fb33fc6db7a459592f21e15977818d0` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:51.041713 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beea3e66-e94b-4a92-92d0-f34fd70c7906 None admin] RBAC: Authorizing `identity:get_user(user_id=6fb33fc6db7a459592f21e15977818d0)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:19:51.043209 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beea3e66-e94b-4a92-92d0-f34fd70c7906 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:19:51.044903 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 547/1096] 10.222.0.49 () {56 vars in 1177 bytes} [Wed Jun 15 10:19:50 2022] GET /identity/v3/users/6fb33fc6db7a459592f21e15977818d0 => generated 285 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 10:19:51.051315 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e76dd3-44f3-40c4-bfd8-b39c600bb3a9 None admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:19:51.155312 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e76dd3-44f3-40c4-bfd8-b39c600bb3a9 None admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:19:51.189246 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e792b7d3-2b5f-4515-9626-49df0fb86945 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:19:51.190110 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e792b7d3-2b5f-4515-9626-49df0fb86945 None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:51.190390 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e792b7d3-2b5f-4515-9626-49df0fb86945 None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:51.190663 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e792b7d3-2b5f-4515-9626-49df0fb86945 None admin] PATH_INFO: `/v3/projects/c0fb512b3149496cb3113317113a0fe1` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:51.191889 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e792b7d3-2b5f-4515-9626-49df0fb86945 None admin] RBAC: Authorizing `identity:get_project(project_id=c0fb512b3149496cb3113317113a0fe1)` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:19:51.193355 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e792b7d3-2b5f-4515-9626-49df0fb86945 None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:19:51.195055 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 550/1097] 10.222.0.49 () {56 vars in 1186 bytes} [Wed Jun 15 10:19:51 2022] GET /identity/v3/projects/c0fb512b3149496cb3113317113a0fe1 => generated 311 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:19:51.201160 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beea3e66-e94b-4a92-92d0-f34fd70c7906 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:19:51.283838 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:19:51.304565 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beea3e66-e94b-4a92-92d0-f34fd70c7906 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:19:51.338202 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902b467b-0fc0-4963-8f43-0af64e4b5d44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:19:51.339112 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902b467b-0fc0-4963-8f43-0af64e4b5d44 None admin] REQUEST_METHOD: `PUT` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:51.339395 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902b467b-0fc0-4963-8f43-0af64e4b5d44 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:51.339669 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902b467b-0fc0-4963-8f43-0af64e4b5d44 None admin] PATH_INFO: `/v3/projects/c0fb512b3149496cb3113317113a0fe1/users/6fb33fc6db7a459592f21e15977818d0/roles/8051174e7a7947d5b991ebd793c2e543` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:51.341629 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902b467b-0fc0-4963-8f43-0af64e4b5d44 None admin] RBAC: Authorizing `identity:create_grant(project_id=c0fb512b3149496cb3113317113a0fe1, user_id=6fb33fc6db7a459592f21e15977818d0, role_id=8051174e7a7947d5b991ebd793c2e543)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:19:51.343057 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902b467b-0fc0-4963-8f43-0af64e4b5d44 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:19:51.356399 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 548/1098] 10.222.0.49 () {58 vars in 1439 bytes} [Wed Jun 15 10:19:51 2022] PUT /identity/v3/projects/c0fb512b3149496cb3113317113a0fe1/users/6fb33fc6db7a459592f21e15977818d0/roles/8051174e7a7947d5b991ebd793c2e543 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:19:53.452442 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9035c463-2c9e-46da-83a7-c66a62b17473 None None] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:53.452976 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9035c463-2c9e-46da-83a7-c66a62b17473 None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:53.453095 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9035c463-2c9e-46da-83a7-c66a62b17473 None None] PATH_INFO: `/` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:53.454358 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 551/1099] 10.222.0.49 () {54 vars in 926 bytes} [Wed Jun 15 10:19:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:19:53.459525 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccbc8ec-5f48-4519-a244-b600545f06a0 None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:53.459956 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccbc8ec-5f48-4519-a244-b600545f06a0 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:53.460090 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccbc8ec-5f48-4519-a244-b600545f06a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:53.477010 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-7ccbc8ec-5f48-4519-a244-b600545f06a0 None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:19:53.478228 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-7ccbc8ec-5f48-4519-a244-b600545f06a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:19:53.544777 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 549/1100] 10.222.0.49 () {58 vars in 1023 bytes} [Wed Jun 15 10:19:53 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 10:19:53.568057 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9035c463-2c9e-46da-83a7-c66a62b17473 None None] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:19:53.579744 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=100 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:19:53.671040 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9035c463-2c9e-46da-83a7-c66a62b17473 None None] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:19:53.703239 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c05a67-e691-4d77-8411-d2e538b370d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:19:53.703959 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c05a67-e691-4d77-8411-d2e538b370d9 None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:53.704241 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c05a67-e691-4d77-8411-d2e538b370d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:53.704515 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c05a67-e691-4d77-8411-d2e538b370d9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:53.715413 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c05a67-e691-4d77-8411-d2e538b370d9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 10:19:53.722871 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application [None req-94c05a67-e691-4d77-8411-d2e538b370d9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 10:19:53.722871 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:19:53.722871 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:19:53.722871 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:19:53.722871 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:19:53.722871 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:19:53.722871 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:19:53.722871 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:19:53.722871 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:19:53.722871 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:19:53.722871 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:19:53.722871 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:19:53.722871 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 10:19:53.722871 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 10:19:53.722871 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 10:19:53.722871 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 10:19:53.724072 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 10:19:53.724072 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 10:19:53.724072 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 10:19:53.724072 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 10:19:53.724072 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:19:53.724072 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:19:53.724072 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:19:53.724072 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:19:53.724072 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:19:53.724072 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:19:53.724072 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:19:53.724072 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application with Lock( Jun 15 10:19:53.724072 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:19:53.724072 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:19:53.724072 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:19:53.724072 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:19:53.725271 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:19:53.725271 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:19:53.725271 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:19:53.725271 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:19:53.725271 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 10:19:53.725271 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 10:19:53.725271 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 10:19:53.725271 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 10:19:53.725271 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 10:19:53.725271 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 10:19:53.725271 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 10:19:53.725271 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application Jun 15 10:19:53.725271 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 552/1101] 10.222.0.49 () {56 vars in 1096 bytes} [Wed Jun 15 10:19:53 2022] GET /identity/v3/roles/admin => generated 83 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:19:53.730079 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ccbc8ec-5f48-4519-a244-b600545f06a0 None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:19:53.796315 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:19:53.827814 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ccbc8ec-5f48-4519-a244-b600545f06a0 None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:19:53.859429 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1922655-61e0-4014-8af4-d7c84ba646bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:19:53.860095 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1922655-61e0-4014-8af4-d7c84ba646bb None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:53.860379 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1922655-61e0-4014-8af4-d7c84ba646bb None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:53.860657 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1922655-61e0-4014-8af4-d7c84ba646bb None admin] PATH_INFO: `/v3/roles` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:53.861397 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1922655-61e0-4014-8af4-d7c84ba646bb None admin] RBAC: Adding query filter params (name=admin) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:19:53.861702 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1922655-61e0-4014-8af4-d7c84ba646bb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:19:53.863111 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1922655-61e0-4014-8af4-d7c84ba646bb None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:19:53.871493 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 550/1102] 10.222.0.49 () {56 vars in 1099 bytes} [Wed Jun 15 10:19:53 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:19:53.877052 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c05a67-e691-4d77-8411-d2e538b370d9 None admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:19:53.941141 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=100 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:19:53.973792 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c05a67-e691-4d77-8411-d2e538b370d9 None admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:19:54.005736 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e64cd3e-24f7-4a4a-acb7-acd9f24c737f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:19:54.006584 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e64cd3e-24f7-4a4a-acb7-acd9f24c737f None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:54.006863 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e64cd3e-24f7-4a4a-acb7-acd9f24c737f None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:54.007164 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e64cd3e-24f7-4a4a-acb7-acd9f24c737f None admin] PATH_INFO: `/v3/users/6fb33fc6db7a459592f21e15977818d0` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:54.008912 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e64cd3e-24f7-4a4a-acb7-acd9f24c737f None admin] RBAC: Authorizing `identity:get_user(user_id=6fb33fc6db7a459592f21e15977818d0)` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:19:54.010325 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e64cd3e-24f7-4a4a-acb7-acd9f24c737f None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:19:54.011926 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 553/1103] 10.222.0.49 () {56 vars in 1177 bytes} [Wed Jun 15 10:19:53 2022] GET /identity/v3/users/6fb33fc6db7a459592f21e15977818d0 => generated 285 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:19:54.017515 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1922655-61e0-4014-8af4-d7c84ba646bb None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:19:54.112115 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1922655-61e0-4014-8af4-d7c84ba646bb None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:19:54.140277 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:19:54.145323 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a33006c-a896-48ea-9c7e-d83c3a1f7348 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:19:54.145959 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a33006c-a896-48ea-9c7e-d83c3a1f7348 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:54.146241 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a33006c-a896-48ea-9c7e-d83c3a1f7348 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:54.146516 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a33006c-a896-48ea-9c7e-d83c3a1f7348 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:54.210524 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a33006c-a896-48ea-9c7e-d83c3a1f7348 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:19:54.211992 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a33006c-a896-48ea-9c7e-d83c3a1f7348 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:19:54.305717 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 551/1104] 10.222.0.49 () {58 vars in 1282 bytes} [Wed Jun 15 10:19:54 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 289 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 15 10:19:54.311342 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e64cd3e-24f7-4a4a-acb7-acd9f24c737f None admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:19:54.407779 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e64cd3e-24f7-4a4a-acb7-acd9f24c737f None admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:19:54.441683 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb14f11-5df4-450b-aefd-66dd7bda0bd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:19:54.442533 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb14f11-5df4-450b-aefd-66dd7bda0bd1 None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:54.442816 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb14f11-5df4-450b-aefd-66dd7bda0bd1 None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:54.443125 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb14f11-5df4-450b-aefd-66dd7bda0bd1 None admin] PATH_INFO: `/v3/projects/c0fb512b3149496cb3113317113a0fe1` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:54.444283 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb14f11-5df4-450b-aefd-66dd7bda0bd1 None admin] RBAC: Authorizing `identity:get_project(project_id=c0fb512b3149496cb3113317113a0fe1)` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:19:54.445729 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb14f11-5df4-450b-aefd-66dd7bda0bd1 None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:19:54.447444 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 554/1105] 10.222.0.49 () {56 vars in 1186 bytes} [Wed Jun 15 10:19:54 2022] GET /identity/v3/projects/c0fb512b3149496cb3113317113a0fe1 => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:19:54.453237 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a33006c-a896-48ea-9c7e-d83c3a1f7348 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:19:54.482377 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:19:54.548611 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a33006c-a896-48ea-9c7e-d83c3a1f7348 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:19:54.580493 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d794830-4800-414c-86ad-ad6e32fb2184 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:19:54.581075 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d794830-4800-414c-86ad-ad6e32fb2184 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:54.581355 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d794830-4800-414c-86ad-ad6e32fb2184 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:54.581627 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d794830-4800-414c-86ad-ad6e32fb2184 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:54.582446 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d794830-4800-414c-86ad-ad6e32fb2184 None admin] RBAC: Adding query filter params (role.id=8051174e7a7947d5b991ebd793c2e543, scope.project.id=c0fb512b3149496cb3113317113a0fe1, user.id=6fb33fc6db7a459592f21e15977818d0) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:19:54.582752 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d794830-4800-414c-86ad-ad6e32fb2184 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:19:54.584197 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d794830-4800-414c-86ad-ad6e32fb2184 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:19:54.590287 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 552/1106] 10.222.0.49 () {56 vars in 1374 bytes} [Wed Jun 15 10:19:54 2022] GET /identity/v3/role_assignments?user.id=6fb33fc6db7a459592f21e15977818d0&scope.project.id=c0fb512b3149496cb3113317113a0fe1&role.id=8051174e7a7947d5b991ebd793c2e543 => generated 608 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:19:56.697805 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d741375-9f26-45ef-9f28-39ab8ed4c224 None None] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:56.698352 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d741375-9f26-45ef-9f28-39ab8ed4c224 None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:56.698352 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d741375-9f26-45ef-9f28-39ab8ed4c224 None None] PATH_INFO: `/` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:56.699745 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 555/1107] 10.222.0.49 () {54 vars in 926 bytes} [Wed Jun 15 10:19:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:19:56.704969 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1dab48-5b9d-4776-9fd2-68ce6b4f1827 None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:56.705356 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1dab48-5b9d-4776-9fd2-68ce6b4f1827 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:56.705528 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1dab48-5b9d-4776-9fd2-68ce6b4f1827 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:56.722496 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-0a1dab48-5b9d-4776-9fd2-68ce6b4f1827 None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:19:56.723719 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-0a1dab48-5b9d-4776-9fd2-68ce6b4f1827 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:19:56.788838 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 553/1108] 10.222.0.49 () {58 vars in 1023 bytes} [Wed Jun 15 10:19:56 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 10:19:56.813295 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d741375-9f26-45ef-9f28-39ab8ed4c224 None None] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:19:56.817073 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=100 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:19:56.911059 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d741375-9f26-45ef-9f28-39ab8ed4c224 None None] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:19:56.943799 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28da48fe-da72-44b0-84e7-58d9f085088c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:19:56.944574 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28da48fe-da72-44b0-84e7-58d9f085088c None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:56.944857 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28da48fe-da72-44b0-84e7-58d9f085088c None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:56.945129 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28da48fe-da72-44b0-84e7-58d9f085088c None admin] PATH_INFO: `/v3/domains/default` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:56.946316 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28da48fe-da72-44b0-84e7-58d9f085088c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:19:56.947774 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28da48fe-da72-44b0-84e7-58d9f085088c None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:19:56.949426 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 556/1109] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Jun 15 10:19:56 2022] GET /identity/v3/domains/default => generated 199 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:19:56.954948 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1dab48-5b9d-4776-9fd2-68ce6b4f1827 None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:19:56.981551 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:19:57.052486 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1dab48-5b9d-4776-9fd2-68ce6b4f1827 None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:19:57.084509 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2ee5bc-6964-451d-804f-c6e171102618 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:19:57.085166 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2ee5bc-6964-451d-804f-c6e171102618 None admin] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:57.085453 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2ee5bc-6964-451d-804f-c6e171102618 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:57.085733 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2ee5bc-6964-451d-804f-c6e171102618 None admin] PATH_INFO: `/v3/users` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:57.086966 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b2ee5bc-6964-451d-804f-c6e171102618 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:19:57.088412 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b2ee5bc-6964-451d-804f-c6e171102618 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:19:57.108190 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 554/1110] 10.222.0.49 () {60 vars in 1132 bytes} [Wed Jun 15 10:19:56 2022] POST /identity/v3/users => generated 286 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 10:19:59.225161 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2420c3-7900-4e4f-82ae-3dfc8107125f None None] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:59.225694 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2420c3-7900-4e4f-82ae-3dfc8107125f None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:59.225694 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2420c3-7900-4e4f-82ae-3dfc8107125f None None] PATH_INFO: `/` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:59.227670 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 557/1111] 10.222.0.49 () {54 vars in 926 bytes} [Wed Jun 15 10:19:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:19:59.232949 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e122c919-cc59-47f0-8477-ab3ac56e1aa2 None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:59.233355 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e122c919-cc59-47f0-8477-ab3ac56e1aa2 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:59.233513 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e122c919-cc59-47f0-8477-ab3ac56e1aa2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:59.249816 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-e122c919-cc59-47f0-8477-ab3ac56e1aa2 None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:19:59.251018 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-e122c919-cc59-47f0-8477-ab3ac56e1aa2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:19:59.315379 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 555/1112] 10.222.0.49 () {58 vars in 1023 bytes} [Wed Jun 15 10:19:59 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 10:19:59.338738 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2420c3-7900-4e4f-82ae-3dfc8107125f None None] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:19:59.407554 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=100 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:19:59.437378 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2420c3-7900-4e4f-82ae-3dfc8107125f None None] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:19:59.470487 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2dcff5-4893-4ded-9935-359f8510f6ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:19:59.471360 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2dcff5-4893-4ded-9935-359f8510f6ac None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:59.471643 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2dcff5-4893-4ded-9935-359f8510f6ac None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:59.471921 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2dcff5-4893-4ded-9935-359f8510f6ac None admin] PATH_INFO: `/v3/roles/4d2118569d074a288b37d120110c300d` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:59.473392 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b2dcff5-4893-4ded-9935-359f8510f6ac None admin] RBAC: Authorizing `identity:get_role(role_id=4d2118569d074a288b37d120110c300d)` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:19:59.474758 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b2dcff5-4893-4ded-9935-359f8510f6ac None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:19:59.476182 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 558/1113] 10.222.0.49 () {56 vars in 1177 bytes} [Wed Jun 15 10:19:59 2022] GET /identity/v3/roles/4d2118569d074a288b37d120110c300d => generated 223 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:19:59.481686 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e122c919-cc59-47f0-8477-ab3ac56e1aa2 None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:19:59.496366 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:19:59.579761 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e122c919-cc59-47f0-8477-ab3ac56e1aa2 None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:19:59.612258 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad418784-18fd-436c-baca-95ff9b85d5bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:19:59.613116 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad418784-18fd-436c-baca-95ff9b85d5bd None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:59.613400 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad418784-18fd-436c-baca-95ff9b85d5bd None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:59.613676 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad418784-18fd-436c-baca-95ff9b85d5bd None admin] PATH_INFO: `/v3/users/8201f24be21041218946daa369bf51e0` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:59.626351 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad418784-18fd-436c-baca-95ff9b85d5bd None admin] RBAC: Authorizing `identity:get_user(user_id=8201f24be21041218946daa369bf51e0)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:19:59.627814 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad418784-18fd-436c-baca-95ff9b85d5bd None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:19:59.629431 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 556/1114] 10.222.0.49 () {56 vars in 1177 bytes} [Wed Jun 15 10:19:59 2022] GET /identity/v3/users/8201f24be21041218946daa369bf51e0 => generated 286 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:19:59.635504 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2dcff5-4893-4ded-9935-359f8510f6ac None admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:19:59.730118 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2dcff5-4893-4ded-9935-359f8510f6ac None admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:19:59.761928 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b57734-c6b5-4984-bb3d-64a87d624437 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:19:59.762556 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b57734-c6b5-4984-bb3d-64a87d624437 None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:59.762839 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b57734-c6b5-4984-bb3d-64a87d624437 None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:59.763142 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b57734-c6b5-4984-bb3d-64a87d624437 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:59.772969 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=100 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:19:59.828741 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b57734-c6b5-4984-bb3d-64a87d624437 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:19:59.830182 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b57734-c6b5-4984-bb3d-64a87d624437 None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:19:59.922275 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 559/1115] 10.222.0.49 () {58 vars in 1282 bytes} [Wed Jun 15 10:19:59 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 287 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 15 10:19:59.928463 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad418784-18fd-436c-baca-95ff9b85d5bd None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:19:59.995861 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:20:00.025917 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad418784-18fd-436c-baca-95ff9b85d5bd None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:20:00.058618 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b14c6a-7d02-4174-b6e8-af2f6ecad24d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:20:00.059500 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b14c6a-7d02-4174-b6e8-af2f6ecad24d None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:20:00.059786 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b14c6a-7d02-4174-b6e8-af2f6ecad24d None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:20:00.060063 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b14c6a-7d02-4174-b6e8-af2f6ecad24d None admin] PATH_INFO: `/v3/projects/c0fb512b3149496cb3113317113a0fe1` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:20:00.061239 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b14c6a-7d02-4174-b6e8-af2f6ecad24d None admin] RBAC: Authorizing `identity:get_project(project_id=c0fb512b3149496cb3113317113a0fe1)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:20:00.062678 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b14c6a-7d02-4174-b6e8-af2f6ecad24d None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:20:00.064379 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 557/1116] 10.222.0.49 () {56 vars in 1186 bytes} [Wed Jun 15 10:19:59 2022] GET /identity/v3/projects/c0fb512b3149496cb3113317113a0fe1 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:20:00.070235 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b57734-c6b5-4984-bb3d-64a87d624437 None admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:20:00.109580 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=100 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:20:00.165766 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b57734-c6b5-4984-bb3d-64a87d624437 None admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:20:00.198453 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00109e4-3135-4403-add0-b52448e8c606 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:20:00.199116 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00109e4-3135-4403-add0-b52448e8c606 None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:20:00.199413 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00109e4-3135-4403-add0-b52448e8c606 None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:20:00.199693 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00109e4-3135-4403-add0-b52448e8c606 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:20:00.200507 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a00109e4-3135-4403-add0-b52448e8c606 None admin] RBAC: Adding query filter params (role.id=4d2118569d074a288b37d120110c300d, scope.project.id=c0fb512b3149496cb3113317113a0fe1, user.id=8201f24be21041218946daa369bf51e0) {{(pid=80119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:20:00.200818 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a00109e4-3135-4403-add0-b52448e8c606 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:20:00.202248 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a00109e4-3135-4403-add0-b52448e8c606 None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:20:00.207549 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 560/1117] 10.222.0.49 () {56 vars in 1374 bytes} [Wed Jun 15 10:20:00 2022] GET /identity/v3/role_assignments?user.id=8201f24be21041218946daa369bf51e0&scope.project.id=c0fb512b3149496cb3113317113a0fe1&role.id=4d2118569d074a288b37d120110c300d => generated 259 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:20:02.286835 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0600756-488c-4c2c-ad7f-16e17d66cf73 None None] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:20:02.287398 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0600756-488c-4c2c-ad7f-16e17d66cf73 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:20:02.287502 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0600756-488c-4c2c-ad7f-16e17d66cf73 None None] PATH_INFO: `/` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:20:02.288798 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 558/1118] 10.222.0.49 () {54 vars in 926 bytes} [Wed Jun 15 10:20:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:20:02.294715 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d81ab77-c61f-4f1a-ac48-c33f76e600b5 None None] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:20:02.295164 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d81ab77-c61f-4f1a-ac48-c33f76e600b5 None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:20:02.295302 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d81ab77-c61f-4f1a-ac48-c33f76e600b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:20:02.311877 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.auth.core [None req-4d81ab77-c61f-4f1a-ac48-c33f76e600b5 None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:20:02.313070 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-4d81ab77-c61f-4f1a-ac48-c33f76e600b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:20:02.377148 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 561/1119] 10.222.0.49 () {58 vars in 1023 bytes} [Wed Jun 15 10:20:02 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 10:20:02.400591 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0600756-488c-4c2c-ad7f-16e17d66cf73 None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:20:02.498453 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0600756-488c-4c2c-ad7f-16e17d66cf73 None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:20:02.532187 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786be8d4-b0e3-4137-8a8f-b7b529b0e276 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:20:02.533028 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786be8d4-b0e3-4137-8a8f-b7b529b0e276 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:20:02.533312 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786be8d4-b0e3-4137-8a8f-b7b529b0e276 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:20:02.533584 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786be8d4-b0e3-4137-8a8f-b7b529b0e276 None admin] PATH_INFO: `/v3/roles/4d2118569d074a288b37d120110c300d` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:20:02.535054 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-786be8d4-b0e3-4137-8a8f-b7b529b0e276 None admin] RBAC: Authorizing `identity:get_role(role_id=4d2118569d074a288b37d120110c300d)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:20:02.536467 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-786be8d4-b0e3-4137-8a8f-b7b529b0e276 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:20:02.537872 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 559/1120] 10.222.0.49 () {56 vars in 1177 bytes} [Wed Jun 15 10:20:02 2022] GET /identity/v3/roles/4d2118569d074a288b37d120110c300d => generated 223 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:20:02.543430 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d81ab77-c61f-4f1a-ac48-c33f76e600b5 None None] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:20:02.575765 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=100 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:20:02.638663 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d81ab77-c61f-4f1a-ac48-c33f76e600b5 None None] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:20:02.670765 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e237eda-6542-480b-b9e9-fdb7eee0d42f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:20:02.671655 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e237eda-6542-480b-b9e9-fdb7eee0d42f None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:20:02.671939 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e237eda-6542-480b-b9e9-fdb7eee0d42f None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:20:02.672212 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e237eda-6542-480b-b9e9-fdb7eee0d42f None admin] PATH_INFO: `/v3/users/8201f24be21041218946daa369bf51e0` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:20:02.673376 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e237eda-6542-480b-b9e9-fdb7eee0d42f None admin] RBAC: Authorizing `identity:get_user(user_id=8201f24be21041218946daa369bf51e0)` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:20:02.675076 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e237eda-6542-480b-b9e9-fdb7eee0d42f None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:20:02.676683 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 562/1121] 10.222.0.49 () {56 vars in 1177 bytes} [Wed Jun 15 10:20:02 2022] GET /identity/v3/users/8201f24be21041218946daa369bf51e0 => generated 286 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:20:02.682464 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786be8d4-b0e3-4137-8a8f-b7b529b0e276 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:20:02.690939 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:20:02.779002 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786be8d4-b0e3-4137-8a8f-b7b529b0e276 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:20:02.811762 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d60110-b32e-4f5f-8f39-332e9990b4f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:20:02.812614 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d60110-b32e-4f5f-8f39-332e9990b4f7 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:20:02.812894 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d60110-b32e-4f5f-8f39-332e9990b4f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:20:02.813166 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d60110-b32e-4f5f-8f39-332e9990b4f7 None admin] PATH_INFO: `/v3/projects/c0fb512b3149496cb3113317113a0fe1` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:20:02.814322 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d60110-b32e-4f5f-8f39-332e9990b4f7 None admin] RBAC: Authorizing `identity:get_project(project_id=c0fb512b3149496cb3113317113a0fe1)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:20:02.815770 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d60110-b32e-4f5f-8f39-332e9990b4f7 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:20:02.817445 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 560/1122] 10.222.0.49 () {56 vars in 1186 bytes} [Wed Jun 15 10:20:02 2022] GET /identity/v3/projects/c0fb512b3149496cb3113317113a0fe1 => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:20:02.823868 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e237eda-6542-480b-b9e9-fdb7eee0d42f None admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:20:02.917517 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e237eda-6542-480b-b9e9-fdb7eee0d42f None admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:20:02.918998 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=100 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:20:02.952320 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15e1636-036c-4903-8689-badcd42ed4cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:20:02.953185 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15e1636-036c-4903-8689-badcd42ed4cc None admin] REQUEST_METHOD: `PUT` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:20:02.953461 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15e1636-036c-4903-8689-badcd42ed4cc None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:20:02.953732 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15e1636-036c-4903-8689-badcd42ed4cc None admin] PATH_INFO: `/v3/projects/c0fb512b3149496cb3113317113a0fe1/users/8201f24be21041218946daa369bf51e0/roles/4d2118569d074a288b37d120110c300d` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:20:02.955910 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c15e1636-036c-4903-8689-badcd42ed4cc None admin] RBAC: Authorizing `identity:create_grant(project_id=c0fb512b3149496cb3113317113a0fe1, user_id=8201f24be21041218946daa369bf51e0, role_id=4d2118569d074a288b37d120110c300d)` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:20:02.957330 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c15e1636-036c-4903-8689-badcd42ed4cc None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:20:02.969224 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 563/1123] 10.222.0.49 () {58 vars in 1439 bytes} [Wed Jun 15 10:20:02 2022] PUT /identity/v3/projects/c0fb512b3149496cb3113317113a0fe1/users/8201f24be21041218946daa369bf51e0/roles/4d2118569d074a288b37d120110c300d => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:20:05.011907 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f6206c-f283-4947-bb15-19fcaecf5aac None None] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:20:05.012461 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f6206c-f283-4947-bb15-19fcaecf5aac None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:20:05.012461 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f6206c-f283-4947-bb15-19fcaecf5aac None None] PATH_INFO: `/` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:20:05.013834 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 561/1124] 10.222.0.49 () {54 vars in 926 bytes} [Wed Jun 15 10:20:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:20:05.019222 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed111bc-01a3-4eff-9fe9-70ea758e18c0 None None] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:20:05.019586 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed111bc-01a3-4eff-9fe9-70ea758e18c0 None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:20:05.019777 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed111bc-01a3-4eff-9fe9-70ea758e18c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:20:05.036125 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.auth.core [None req-3ed111bc-01a3-4eff-9fe9-70ea758e18c0 None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:20:05.037365 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-3ed111bc-01a3-4eff-9fe9-70ea758e18c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:20:05.102336 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 564/1125] 10.222.0.49 () {58 vars in 1023 bytes} [Wed Jun 15 10:20:05 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 10:20:05.125372 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f6206c-f283-4947-bb15-19fcaecf5aac None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:20:05.198962 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:20:05.224530 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f6206c-f283-4947-bb15-19fcaecf5aac None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:20:05.262120 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446bafc7-c6c9-44e7-8176-eefe7ab20938 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:20:05.263270 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446bafc7-c6c9-44e7-8176-eefe7ab20938 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:20:05.263558 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446bafc7-c6c9-44e7-8176-eefe7ab20938 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:20:05.263835 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446bafc7-c6c9-44e7-8176-eefe7ab20938 None admin] PATH_INFO: `/v3/roles/4d2118569d074a288b37d120110c300d` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:20:05.265313 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446bafc7-c6c9-44e7-8176-eefe7ab20938 None admin] RBAC: Authorizing `identity:get_role(role_id=4d2118569d074a288b37d120110c300d)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:20:05.266675 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446bafc7-c6c9-44e7-8176-eefe7ab20938 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:20:05.268066 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 562/1126] 10.222.0.49 () {56 vars in 1177 bytes} [Wed Jun 15 10:20:05 2022] GET /identity/v3/roles/4d2118569d074a288b37d120110c300d => generated 223 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:20:05.274304 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed111bc-01a3-4eff-9fe9-70ea758e18c0 None None] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:20:05.362195 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:20:05.371545 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed111bc-01a3-4eff-9fe9-70ea758e18c0 None None] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:20:05.404324 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0087727-2e2a-4eee-99fd-40408b7e3dc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:20:05.405201 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0087727-2e2a-4eee-99fd-40408b7e3dc7 None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:20:05.405487 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0087727-2e2a-4eee-99fd-40408b7e3dc7 None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:20:05.405764 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0087727-2e2a-4eee-99fd-40408b7e3dc7 None admin] PATH_INFO: `/v3/users/8201f24be21041218946daa369bf51e0` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:20:05.406941 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0087727-2e2a-4eee-99fd-40408b7e3dc7 None admin] RBAC: Authorizing `identity:get_user(user_id=8201f24be21041218946daa369bf51e0)` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:20:05.408410 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0087727-2e2a-4eee-99fd-40408b7e3dc7 None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:20:05.410016 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 565/1127] 10.222.0.49 () {56 vars in 1177 bytes} [Wed Jun 15 10:20:05 2022] GET /identity/v3/users/8201f24be21041218946daa369bf51e0 => generated 286 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:20:05.415619 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446bafc7-c6c9-44e7-8176-eefe7ab20938 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:20:05.510775 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446bafc7-c6c9-44e7-8176-eefe7ab20938 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:20:05.544631 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7219bc4-a8aa-4f97-bb58-bd0fa93703bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:20:05.545268 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7219bc4-a8aa-4f97-bb58-bd0fa93703bc None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:20:05.545555 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7219bc4-a8aa-4f97-bb58-bd0fa93703bc None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:20:05.545832 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7219bc4-a8aa-4f97-bb58-bd0fa93703bc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:20:05.557353 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:20:05.611339 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7219bc4-a8aa-4f97-bb58-bd0fa93703bc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:20:05.612859 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7219bc4-a8aa-4f97-bb58-bd0fa93703bc None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:20:05.706721 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 563/1128] 10.222.0.49 () {58 vars in 1282 bytes} [Wed Jun 15 10:20:05 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 15 10:20:05.713022 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0087727-2e2a-4eee-99fd-40408b7e3dc7 None admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:20:05.809695 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0087727-2e2a-4eee-99fd-40408b7e3dc7 None admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:20:05.842087 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9aba57-63c8-4d19-86d9-dc6120162df7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:20:05.842941 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9aba57-63c8-4d19-86d9-dc6120162df7 None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:20:05.843254 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9aba57-63c8-4d19-86d9-dc6120162df7 None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:20:05.843534 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9aba57-63c8-4d19-86d9-dc6120162df7 None admin] PATH_INFO: `/v3/projects/c0fb512b3149496cb3113317113a0fe1` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:20:05.844704 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9aba57-63c8-4d19-86d9-dc6120162df7 None admin] RBAC: Authorizing `identity:get_project(project_id=c0fb512b3149496cb3113317113a0fe1)` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:20:05.846144 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9aba57-63c8-4d19-86d9-dc6120162df7 None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:20:05.847849 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 566/1129] 10.222.0.49 () {56 vars in 1186 bytes} [Wed Jun 15 10:20:05 2022] GET /identity/v3/projects/c0fb512b3149496cb3113317113a0fe1 => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:20:05.853888 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7219bc4-a8aa-4f97-bb58-bd0fa93703bc None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:20:05.895076 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:20:05.950169 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7219bc4-a8aa-4f97-bb58-bd0fa93703bc None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:20:05.981962 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097e1d05-7962-4e87-bf0f-9c6d22cbe940 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:20:05.982596 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097e1d05-7962-4e87-bf0f-9c6d22cbe940 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:20:05.982879 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097e1d05-7962-4e87-bf0f-9c6d22cbe940 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:20:05.983181 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097e1d05-7962-4e87-bf0f-9c6d22cbe940 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:20:05.983999 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-097e1d05-7962-4e87-bf0f-9c6d22cbe940 None admin] RBAC: Adding query filter params (role.id=4d2118569d074a288b37d120110c300d, scope.project.id=c0fb512b3149496cb3113317113a0fe1, user.id=8201f24be21041218946daa369bf51e0) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:20:05.984307 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-097e1d05-7962-4e87-bf0f-9c6d22cbe940 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:20:05.985708 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-097e1d05-7962-4e87-bf0f-9c6d22cbe940 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:20:05.991358 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 564/1130] 10.222.0.49 () {56 vars in 1374 bytes} [Wed Jun 15 10:20:05 2022] GET /identity/v3/role_assignments?user.id=8201f24be21041218946daa369bf51e0&scope.project.id=c0fb512b3149496cb3113317113a0fe1&role.id=4d2118569d074a288b37d120110c300d => generated 608 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:20:08.100439 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ec8d7c-6219-409a-b675-6c3ac39b21fa None None] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:20:08.100954 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ec8d7c-6219-409a-b675-6c3ac39b21fa None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:20:08.100954 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ec8d7c-6219-409a-b675-6c3ac39b21fa None None] PATH_INFO: `/` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:20:08.102344 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 567/1131] 10.222.0.49 () {54 vars in 926 bytes} [Wed Jun 15 10:20:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:20:08.107455 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34624299-7f21-4180-9995-d7fb7b73b6ef None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:20:08.107892 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34624299-7f21-4180-9995-d7fb7b73b6ef None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:20:08.108017 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34624299-7f21-4180-9995-d7fb7b73b6ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:20:08.125441 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-34624299-7f21-4180-9995-d7fb7b73b6ef None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:20:08.126631 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-34624299-7f21-4180-9995-d7fb7b73b6ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:20:08.190165 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 565/1132] 10.222.0.49 () {58 vars in 1023 bytes} [Wed Jun 15 10:20:08 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 10:20:08.213726 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ec8d7c-6219-409a-b675-6c3ac39b21fa None None] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:20:08.241232 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=100 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:20:08.312404 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ec8d7c-6219-409a-b675-6c3ac39b21fa None None] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:20:08.344342 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823eba52-98a2-4d68-a29a-7d2f8ebba866 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:20:08.345120 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823eba52-98a2-4d68-a29a-7d2f8ebba866 None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:20:08.345403 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823eba52-98a2-4d68-a29a-7d2f8ebba866 None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:20:08.345678 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823eba52-98a2-4d68-a29a-7d2f8ebba866 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:20:08.346838 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-823eba52-98a2-4d68-a29a-7d2f8ebba866 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:20:08.348303 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-823eba52-98a2-4d68-a29a-7d2f8ebba866 None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:20:08.349885 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 568/1133] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Jun 15 10:20:08 2022] GET /identity/v3/domains/default => generated 199 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:20:08.355536 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34624299-7f21-4180-9995-d7fb7b73b6ef None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:20:08.388941 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:20:08.450208 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34624299-7f21-4180-9995-d7fb7b73b6ef None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:20:08.482261 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27bc40ae-1c8c-487e-83e4-f112407c8523 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:20:08.482900 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bc40ae-1c8c-487e-83e4-f112407c8523 None admin] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:20:08.483208 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bc40ae-1c8c-487e-83e4-f112407c8523 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:20:08.483484 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bc40ae-1c8c-487e-83e4-f112407c8523 None admin] PATH_INFO: `/v3/projects` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:20:08.484169 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27bc40ae-1c8c-487e-83e4-f112407c8523 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:20:08.485574 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27bc40ae-1c8c-487e-83e4-f112407c8523 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:20:08.509764 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 566/1134] 10.222.0.49 () {60 vars in 1141 bytes} [Wed Jun 15 10:20:08 2022] POST /identity/v3/projects => generated 311 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 10:20:10.639912 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86c644c-c825-43bd-ac4e-25cd6d7b23b9 None None] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:20:10.640435 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86c644c-c825-43bd-ac4e-25cd6d7b23b9 None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:20:10.640545 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86c644c-c825-43bd-ac4e-25cd6d7b23b9 None None] PATH_INFO: `/` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:20:10.641819 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 569/1135] 10.222.0.49 () {54 vars in 926 bytes} [Wed Jun 15 10:20:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:20:10.647070 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb41aec-465b-4538-a8e6-d36d092cffea None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:20:10.647499 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb41aec-465b-4538-a8e6-d36d092cffea None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:20:10.647672 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb41aec-465b-4538-a8e6-d36d092cffea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:20:10.664651 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-cbb41aec-465b-4538-a8e6-d36d092cffea None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:20:10.665849 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-cbb41aec-465b-4538-a8e6-d36d092cffea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:20:10.730723 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 567/1136] 10.222.0.49 () {58 vars in 1023 bytes} [Wed Jun 15 10:20:10 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 10:20:10.754824 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86c644c-c825-43bd-ac4e-25cd6d7b23b9 None None] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:20:10.848247 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=100 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:20:10.855388 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86c644c-c825-43bd-ac4e-25cd6d7b23b9 None None] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:20:10.888167 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735d9b24-fc46-4d79-8fa1-0408d936ba34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:20:10.888950 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735d9b24-fc46-4d79-8fa1-0408d936ba34 None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:20:10.889230 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735d9b24-fc46-4d79-8fa1-0408d936ba34 None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:20:10.889500 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735d9b24-fc46-4d79-8fa1-0408d936ba34 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:20:10.890695 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-735d9b24-fc46-4d79-8fa1-0408d936ba34 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:20:10.892107 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-735d9b24-fc46-4d79-8fa1-0408d936ba34 None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:20:10.893700 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 570/1137] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Jun 15 10:20:10 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:20:10.899934 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb41aec-465b-4538-a8e6-d36d092cffea None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:20:10.917357 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:20:10.995666 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb41aec-465b-4538-a8e6-d36d092cffea None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:20:11.030616 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0c6d54-cccf-4797-a2cd-daa56dc3be32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:20:11.031315 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0c6d54-cccf-4797-a2cd-daa56dc3be32 None admin] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:20:11.031603 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0c6d54-cccf-4797-a2cd-daa56dc3be32 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:20:11.031882 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0c6d54-cccf-4797-a2cd-daa56dc3be32 None admin] PATH_INFO: `/v3/users` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:20:11.032601 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a0c6d54-cccf-4797-a2cd-daa56dc3be32 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:20:11.034035 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a0c6d54-cccf-4797-a2cd-daa56dc3be32 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:20:11.053299 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 568/1138] 10.222.0.49 () {60 vars in 1132 bytes} [Wed Jun 15 10:20:10 2022] POST /identity/v3/users => generated 286 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 10:20:13.189614 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e4887f-32fc-4526-b9b5-47caca1d3a61 None None] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:20:13.190119 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e4887f-32fc-4526-b9b5-47caca1d3a61 None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:20:13.190119 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e4887f-32fc-4526-b9b5-47caca1d3a61 None None] PATH_INFO: `/` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:20:13.191496 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 571/1139] 10.222.0.49 () {54 vars in 926 bytes} [Wed Jun 15 10:20:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:20:13.196627 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1feffc86-dbfe-4818-9013-ba95fabea4bb None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:20:13.197062 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1feffc86-dbfe-4818-9013-ba95fabea4bb None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:20:13.197188 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1feffc86-dbfe-4818-9013-ba95fabea4bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:20:13.213898 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-1feffc86-dbfe-4818-9013-ba95fabea4bb None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:20:13.215118 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-1feffc86-dbfe-4818-9013-ba95fabea4bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:20:13.279776 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 569/1140] 10.222.0.49 () {58 vars in 1023 bytes} [Wed Jun 15 10:20:13 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 10:20:13.303162 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e4887f-32fc-4526-b9b5-47caca1d3a61 None None] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:20:13.401984 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e4887f-32fc-4526-b9b5-47caca1d3a61 None None] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:20:13.434523 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6aaf410-e3d8-4276-b500-499acdec449a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:20:13.435332 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6aaf410-e3d8-4276-b500-499acdec449a None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:20:13.435619 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6aaf410-e3d8-4276-b500-499acdec449a None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:20:13.435898 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6aaf410-e3d8-4276-b500-499acdec449a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:20:13.447591 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6aaf410-e3d8-4276-b500-499acdec449a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 10:20:13.455110 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application [None req-a6aaf410-e3d8-4276-b500-499acdec449a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 10:20:13.455110 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:20:13.455110 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:20:13.455110 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:20:13.455110 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:20:13.455110 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:20:13.455110 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:20:13.455110 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:20:13.455110 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:20:13.455110 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:20:13.455110 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:20:13.455110 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:20:13.455110 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 10:20:13.455110 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 10:20:13.455110 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 10:20:13.455110 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 10:20:13.456197 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 10:20:13.456197 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 10:20:13.456197 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 10:20:13.456197 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 10:20:13.456197 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:20:13.456197 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:20:13.456197 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:20:13.456197 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:20:13.456197 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:20:13.456197 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:20:13.456197 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:20:13.456197 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application with Lock( Jun 15 10:20:13.456197 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:20:13.456197 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:20:13.456197 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:20:13.456197 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:20:13.457126 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:20:13.457126 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:20:13.457126 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:20:13.457126 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:20:13.457126 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 10:20:13.457126 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 10:20:13.457126 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 10:20:13.457126 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 10:20:13.457126 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 10:20:13.457126 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 10:20:13.457126 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 10:20:13.457126 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application Jun 15 10:20:13.457126 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 572/1141] 10.222.0.49 () {56 vars in 1096 bytes} [Wed Jun 15 10:20:13 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:20:13.462219 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1feffc86-dbfe-4818-9013-ba95fabea4bb None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:20:13.465300 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:20:13.557396 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1feffc86-dbfe-4818-9013-ba95fabea4bb None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:20:13.589520 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94748a9-8af8-4658-aa4c-5503b0c05278 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:20:13.590167 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94748a9-8af8-4658-aa4c-5503b0c05278 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:20:13.590447 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94748a9-8af8-4658-aa4c-5503b0c05278 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:20:13.590719 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94748a9-8af8-4658-aa4c-5503b0c05278 None admin] PATH_INFO: `/v3/roles` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:20:13.591486 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94748a9-8af8-4658-aa4c-5503b0c05278 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:20:13.591789 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94748a9-8af8-4658-aa4c-5503b0c05278 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:20:13.593142 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94748a9-8af8-4658-aa4c-5503b0c05278 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:20:13.600816 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 570/1142] 10.222.0.49 () {56 vars in 1099 bytes} [Wed Jun 15 10:20:13 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:20:13.606499 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6aaf410-e3d8-4276-b500-499acdec449a None admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:20:13.631550 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=100 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:20:13.709864 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6aaf410-e3d8-4276-b500-499acdec449a None admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:20:13.742757 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da685734-a7bf-466d-8b74-a7320b82183b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:20:13.743645 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da685734-a7bf-466d-8b74-a7320b82183b None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:20:13.743929 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da685734-a7bf-466d-8b74-a7320b82183b None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:20:13.744206 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da685734-a7bf-466d-8b74-a7320b82183b None admin] PATH_INFO: `/v3/users/16ebd3f590f9472580cfb0d08522d235` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:20:13.757010 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da685734-a7bf-466d-8b74-a7320b82183b None admin] RBAC: Authorizing `identity:get_user(user_id=16ebd3f590f9472580cfb0d08522d235)` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:20:13.758480 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da685734-a7bf-466d-8b74-a7320b82183b None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:20:13.760134 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 573/1143] 10.222.0.49 () {56 vars in 1177 bytes} [Wed Jun 15 10:20:13 2022] GET /identity/v3/users/16ebd3f590f9472580cfb0d08522d235 => generated 286 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:20:13.765706 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94748a9-8af8-4658-aa4c-5503b0c05278 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:20:13.829378 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:20:13.862558 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94748a9-8af8-4658-aa4c-5503b0c05278 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:20:13.894247 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d81b7b-4f09-408b-990e-dacccb3d1159 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:20:13.894883 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d81b7b-4f09-408b-990e-dacccb3d1159 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:20:13.895192 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d81b7b-4f09-408b-990e-dacccb3d1159 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:20:13.895465 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d81b7b-4f09-408b-990e-dacccb3d1159 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:20:13.958892 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d81b7b-4f09-408b-990e-dacccb3d1159 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:20:13.961677 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d81b7b-4f09-408b-990e-dacccb3d1159 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:20:14.021623 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:20:14.056510 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 571/1144] 10.222.0.49 () {58 vars in 1282 bytes} [Wed Jun 15 10:20:13 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 291 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 15 10:20:14.062313 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da685734-a7bf-466d-8b74-a7320b82183b None admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:20:14.140498 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=100 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:20:14.158749 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da685734-a7bf-466d-8b74-a7320b82183b None admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:20:14.191631 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18316fa-485b-46aa-b9cb-fe33acbfe5fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:20:14.192481 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18316fa-485b-46aa-b9cb-fe33acbfe5fa None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:20:14.192765 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18316fa-485b-46aa-b9cb-fe33acbfe5fa None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:20:14.193039 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18316fa-485b-46aa-b9cb-fe33acbfe5fa None admin] PATH_INFO: `/v3/projects/b1a5ca7891904568918c83e761b1038b` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:20:14.194203 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18316fa-485b-46aa-b9cb-fe33acbfe5fa None admin] RBAC: Authorizing `identity:get_project(project_id=b1a5ca7891904568918c83e761b1038b)` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:20:14.195730 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18316fa-485b-46aa-b9cb-fe33acbfe5fa None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:20:14.197392 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 574/1145] 10.222.0.49 () {56 vars in 1186 bytes} [Wed Jun 15 10:20:14 2022] GET /identity/v3/projects/b1a5ca7891904568918c83e761b1038b => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:20:14.203240 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d81b7b-4f09-408b-990e-dacccb3d1159 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:20:14.297552 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d81b7b-4f09-408b-990e-dacccb3d1159 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:20:14.329268 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6307eb2-d961-402b-96b5-27ae64f87e16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:20:14.329903 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6307eb2-d961-402b-96b5-27ae64f87e16 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:20:14.330181 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6307eb2-d961-402b-96b5-27ae64f87e16 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:20:14.330453 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6307eb2-d961-402b-96b5-27ae64f87e16 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:20:14.331294 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6307eb2-d961-402b-96b5-27ae64f87e16 None admin] RBAC: Adding query filter params (role.id=8051174e7a7947d5b991ebd793c2e543, scope.project.id=b1a5ca7891904568918c83e761b1038b, user.id=16ebd3f590f9472580cfb0d08522d235) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:20:14.331601 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6307eb2-d961-402b-96b5-27ae64f87e16 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:20:14.333006 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6307eb2-d961-402b-96b5-27ae64f87e16 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:20:14.338740 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 572/1146] 10.222.0.49 () {56 vars in 1374 bytes} [Wed Jun 15 10:20:14 2022] GET /identity/v3/role_assignments?user.id=16ebd3f590f9472580cfb0d08522d235&scope.project.id=b1a5ca7891904568918c83e761b1038b&role.id=8051174e7a7947d5b991ebd793c2e543 => generated 259 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:20:16.406921 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5a3d78-9539-40e2-a791-b21a831db6cf None None] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:20:16.407527 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5a3d78-9539-40e2-a791-b21a831db6cf None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:20:16.407527 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5a3d78-9539-40e2-a791-b21a831db6cf None None] PATH_INFO: `/` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:20:16.408852 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 575/1147] 10.222.0.49 () {54 vars in 926 bytes} [Wed Jun 15 10:20:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:20:16.413878 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21622c9-e0dc-4ab0-8e99-1b0ecf6e5a73 None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:20:16.414286 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21622c9-e0dc-4ab0-8e99-1b0ecf6e5a73 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:20:16.414434 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21622c9-e0dc-4ab0-8e99-1b0ecf6e5a73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:20:16.431501 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-c21622c9-e0dc-4ab0-8e99-1b0ecf6e5a73 None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:20:16.432687 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-c21622c9-e0dc-4ab0-8e99-1b0ecf6e5a73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:20:16.455767 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:20:16.497082 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 573/1148] 10.222.0.49 () {58 vars in 1023 bytes} [Wed Jun 15 10:20:16 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 10:20:16.520044 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5a3d78-9539-40e2-a791-b21a831db6cf None None] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:20:16.617237 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5a3d78-9539-40e2-a791-b21a831db6cf None None] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:20:16.649454 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0c0f22-3178-41be-b10f-cc533d50161f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:20:16.650230 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0c0f22-3178-41be-b10f-cc533d50161f None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:20:16.650511 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0c0f22-3178-41be-b10f-cc533d50161f None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:20:16.650779 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0c0f22-3178-41be-b10f-cc533d50161f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:20:16.660523 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd0c0f22-3178-41be-b10f-cc533d50161f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 10:20:16.668474 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application [None req-bd0c0f22-3178-41be-b10f-cc533d50161f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 10:20:16.668474 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:20:16.668474 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:20:16.668474 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:20:16.668474 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:20:16.668474 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:20:16.668474 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:20:16.668474 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:20:16.668474 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:20:16.668474 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:20:16.668474 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:20:16.668474 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:20:16.668474 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 10:20:16.668474 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 10:20:16.668474 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 10:20:16.668474 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 10:20:16.669594 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 10:20:16.669594 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 10:20:16.669594 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 10:20:16.669594 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 10:20:16.669594 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:20:16.669594 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:20:16.669594 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:20:16.669594 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:20:16.669594 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:20:16.669594 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:20:16.669594 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:20:16.669594 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application with Lock( Jun 15 10:20:16.669594 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:20:16.669594 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:20:16.669594 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:20:16.669594 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:20:16.670564 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:20:16.670564 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:20:16.670564 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:20:16.670564 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:20:16.670564 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 10:20:16.670564 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 10:20:16.670564 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 10:20:16.670564 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 10:20:16.670564 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 10:20:16.670564 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 10:20:16.670564 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 10:20:16.670564 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application Jun 15 10:20:16.670564 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 576/1149] 10.222.0.49 () {56 vars in 1096 bytes} [Wed Jun 15 10:20:16 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:20:16.675750 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21622c9-e0dc-4ab0-8e99-1b0ecf6e5a73 None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:20:16.769888 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21622c9-e0dc-4ab0-8e99-1b0ecf6e5a73 None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:20:16.802924 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62eadd21-708d-4179-9308-71e57856fa66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:20:16.803616 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62eadd21-708d-4179-9308-71e57856fa66 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:20:16.803901 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62eadd21-708d-4179-9308-71e57856fa66 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:20:16.804722 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62eadd21-708d-4179-9308-71e57856fa66 None admin] PATH_INFO: `/v3/roles` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:20:16.805461 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62eadd21-708d-4179-9308-71e57856fa66 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:20:16.805764 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62eadd21-708d-4179-9308-71e57856fa66 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:20:16.807144 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62eadd21-708d-4179-9308-71e57856fa66 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:20:16.814898 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 574/1150] 10.222.0.49 () {56 vars in 1099 bytes} [Wed Jun 15 10:20:16 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 10:20:16.820480 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0c0f22-3178-41be-b10f-cc533d50161f None admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:20:16.832937 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=100 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:20:16.916553 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0c0f22-3178-41be-b10f-cc533d50161f None admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:20:16.948493 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eed8f44-2380-4017-8de9-653c85341c6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:20:16.949350 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eed8f44-2380-4017-8de9-653c85341c6d None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:20:16.949630 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eed8f44-2380-4017-8de9-653c85341c6d None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:20:16.949900 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eed8f44-2380-4017-8de9-653c85341c6d None admin] PATH_INFO: `/v3/users/16ebd3f590f9472580cfb0d08522d235` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:20:16.951061 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eed8f44-2380-4017-8de9-653c85341c6d None admin] RBAC: Authorizing `identity:get_user(user_id=16ebd3f590f9472580cfb0d08522d235)` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:20:16.952503 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eed8f44-2380-4017-8de9-653c85341c6d None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:20:16.954079 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 577/1151] 10.222.0.49 () {56 vars in 1177 bytes} [Wed Jun 15 10:20:16 2022] GET /identity/v3/users/16ebd3f590f9472580cfb0d08522d235 => generated 286 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:20:16.959631 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62eadd21-708d-4179-9308-71e57856fa66 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:20:16.983890 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:20:17.056708 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62eadd21-708d-4179-9308-71e57856fa66 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:20:17.088691 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d44d3f-63d2-4f01-9e15-c527c2b0b9c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:20:17.089533 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d44d3f-63d2-4f01-9e15-c527c2b0b9c8 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:20:17.089817 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d44d3f-63d2-4f01-9e15-c527c2b0b9c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:20:17.090091 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d44d3f-63d2-4f01-9e15-c527c2b0b9c8 None admin] PATH_INFO: `/v3/projects/b1a5ca7891904568918c83e761b1038b` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:20:17.091287 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d44d3f-63d2-4f01-9e15-c527c2b0b9c8 None admin] RBAC: Authorizing `identity:get_project(project_id=b1a5ca7891904568918c83e761b1038b)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:20:17.092697 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d44d3f-63d2-4f01-9e15-c527c2b0b9c8 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:20:17.094353 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 575/1152] 10.222.0.49 () {56 vars in 1186 bytes} [Wed Jun 15 10:20:16 2022] GET /identity/v3/projects/b1a5ca7891904568918c83e761b1038b => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:20:17.100180 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eed8f44-2380-4017-8de9-653c85341c6d None admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:20:17.175133 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=100 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:20:17.195498 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eed8f44-2380-4017-8de9-653c85341c6d None admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:20:17.227664 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85782581-37c7-4848-9fa3-a8906b585c0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:20:17.228536 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85782581-37c7-4848-9fa3-a8906b585c0e None admin] REQUEST_METHOD: `PUT` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:20:17.228817 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85782581-37c7-4848-9fa3-a8906b585c0e None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:20:17.229091 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85782581-37c7-4848-9fa3-a8906b585c0e None admin] PATH_INFO: `/v3/projects/b1a5ca7891904568918c83e761b1038b/users/16ebd3f590f9472580cfb0d08522d235/roles/8051174e7a7947d5b991ebd793c2e543` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:20:17.231006 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85782581-37c7-4848-9fa3-a8906b585c0e None admin] RBAC: Authorizing `identity:create_grant(project_id=b1a5ca7891904568918c83e761b1038b, user_id=16ebd3f590f9472580cfb0d08522d235, role_id=8051174e7a7947d5b991ebd793c2e543)` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:20:17.232445 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85782581-37c7-4848-9fa3-a8906b585c0e None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:20:17.245863 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 578/1153] 10.222.0.49 () {58 vars in 1439 bytes} [Wed Jun 15 10:20:17 2022] PUT /identity/v3/projects/b1a5ca7891904568918c83e761b1038b/users/16ebd3f590f9472580cfb0d08522d235/roles/8051174e7a7947d5b991ebd793c2e543 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 10:20:19.324375 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595e99a0-d65e-44a5-b306-8458511a6a3d None None] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:20:19.324903 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595e99a0-d65e-44a5-b306-8458511a6a3d None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:20:19.324903 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595e99a0-d65e-44a5-b306-8458511a6a3d None None] PATH_INFO: `/` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:20:19.326275 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 576/1154] 10.222.0.49 () {54 vars in 926 bytes} [Wed Jun 15 10:20:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:20:19.331355 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5c09b2-4ba8-44cb-93f4-ce92ca81a2bd None None] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:20:19.331793 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5c09b2-4ba8-44cb-93f4-ce92ca81a2bd None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:20:19.331895 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5c09b2-4ba8-44cb-93f4-ce92ca81a2bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:20:19.348831 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.auth.core [None req-ba5c09b2-4ba8-44cb-93f4-ce92ca81a2bd None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:20:19.350003 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-ba5c09b2-4ba8-44cb-93f4-ce92ca81a2bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:20:19.414906 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 579/1155] 10.222.0.49 () {58 vars in 1023 bytes} [Wed Jun 15 10:20:19 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 15 10:20:19.438412 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595e99a0-d65e-44a5-b306-8458511a6a3d None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:20:19.526851 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:20:19.537379 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595e99a0-d65e-44a5-b306-8458511a6a3d None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:20:19.569470 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189fd5f3-ef9a-4036-9350-97e30564eebf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:20:19.570246 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189fd5f3-ef9a-4036-9350-97e30564eebf None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:20:19.570528 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189fd5f3-ef9a-4036-9350-97e30564eebf None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:20:19.570803 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189fd5f3-ef9a-4036-9350-97e30564eebf None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:20:19.580944 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-189fd5f3-ef9a-4036-9350-97e30564eebf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 10:20:19.588836 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application [None req-189fd5f3-ef9a-4036-9350-97e30564eebf None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 10:20:19.588836 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:20:19.588836 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:20:19.588836 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:20:19.588836 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:20:19.588836 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:20:19.588836 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:20:19.588836 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:20:19.588836 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:20:19.588836 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:20:19.588836 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:20:19.588836 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:20:19.588836 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 10:20:19.588836 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 10:20:19.588836 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 10:20:19.588836 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 10:20:19.589963 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 10:20:19.589963 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 10:20:19.589963 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 10:20:19.589963 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 10:20:19.589963 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:20:19.589963 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:20:19.589963 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:20:19.589963 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:20:19.589963 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:20:19.589963 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:20:19.589963 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:20:19.589963 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application with Lock( Jun 15 10:20:19.589963 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:20:19.589963 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:20:19.589963 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:20:19.589963 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:20:19.590976 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:20:19.590976 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:20:19.590976 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:20:19.590976 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:20:19.590976 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 10:20:19.590976 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 10:20:19.590976 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 10:20:19.590976 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 10:20:19.590976 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 10:20:19.590976 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 10:20:19.590976 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 10:20:19.590976 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Jun 15 10:20:19.590976 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 577/1156] 10.222.0.49 () {56 vars in 1096 bytes} [Wed Jun 15 10:20:19 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:20:19.596082 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5c09b2-4ba8-44cb-93f4-ce92ca81a2bd None None] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:20:19.658737 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:20:19.690479 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5c09b2-4ba8-44cb-93f4-ce92ca81a2bd None None] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:20:19.723318 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52943ed-37a5-40ad-a9c8-0d757abc8eea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:20:19.723973 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52943ed-37a5-40ad-a9c8-0d757abc8eea None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:20:19.724254 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52943ed-37a5-40ad-a9c8-0d757abc8eea None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:20:19.724526 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52943ed-37a5-40ad-a9c8-0d757abc8eea None admin] PATH_INFO: `/v3/roles` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:20:19.725253 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52943ed-37a5-40ad-a9c8-0d757abc8eea None admin] RBAC: Adding query filter params (name=admin) {{(pid=80119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:20:19.725553 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52943ed-37a5-40ad-a9c8-0d757abc8eea None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:20:19.726927 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52943ed-37a5-40ad-a9c8-0d757abc8eea None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:20:19.734621 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 580/1157] 10.222.0.49 () {56 vars in 1099 bytes} [Wed Jun 15 10:20:19 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:20:19.740309 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189fd5f3-ef9a-4036-9350-97e30564eebf None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:20:19.837838 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189fd5f3-ef9a-4036-9350-97e30564eebf None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:20:19.869641 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac53a35-556b-4bb7-9ddc-f791b22d12f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:20:19.870758 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac53a35-556b-4bb7-9ddc-f791b22d12f3 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:20:19.871047 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac53a35-556b-4bb7-9ddc-f791b22d12f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:20:19.871350 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac53a35-556b-4bb7-9ddc-f791b22d12f3 None admin] PATH_INFO: `/v3/users/16ebd3f590f9472580cfb0d08522d235` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:20:19.872543 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac53a35-556b-4bb7-9ddc-f791b22d12f3 None admin] RBAC: Authorizing `identity:get_user(user_id=16ebd3f590f9472580cfb0d08522d235)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:20:19.873974 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac53a35-556b-4bb7-9ddc-f791b22d12f3 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:20:19.875622 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 578/1158] 10.222.0.49 () {56 vars in 1177 bytes} [Wed Jun 15 10:20:19 2022] GET /identity/v3/users/16ebd3f590f9472580cfb0d08522d235 => generated 286 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:20:19.881268 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52943ed-37a5-40ad-a9c8-0d757abc8eea None admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:20:19.977411 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52943ed-37a5-40ad-a9c8-0d757abc8eea None admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:20:20.004027 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=100 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:20:20.010526 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e68762a-d9b7-4da9-89d9-02ed97543388 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:20:20.011208 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e68762a-d9b7-4da9-89d9-02ed97543388 None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:20:20.011498 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e68762a-d9b7-4da9-89d9-02ed97543388 None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:20:20.011779 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e68762a-d9b7-4da9-89d9-02ed97543388 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:20:20.077154 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e68762a-d9b7-4da9-89d9-02ed97543388 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:20:20.078604 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e68762a-d9b7-4da9-89d9-02ed97543388 None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:20:20.172038 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 581/1159] 10.222.0.49 () {58 vars in 1282 bytes} [Wed Jun 15 10:20:19 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 291 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 15 10:20:20.177990 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac53a35-556b-4bb7-9ddc-f791b22d12f3 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:20:20.197112 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:20:20.274142 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac53a35-556b-4bb7-9ddc-f791b22d12f3 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:20:20.307435 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eaadd4f-157f-4eb5-8acd-ab750d32c931 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:20:20.308280 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eaadd4f-157f-4eb5-8acd-ab750d32c931 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:20:20.308560 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eaadd4f-157f-4eb5-8acd-ab750d32c931 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:20:20.308831 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eaadd4f-157f-4eb5-8acd-ab750d32c931 None admin] PATH_INFO: `/v3/projects/b1a5ca7891904568918c83e761b1038b` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:20:20.310001 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eaadd4f-157f-4eb5-8acd-ab750d32c931 None admin] RBAC: Authorizing `identity:get_project(project_id=b1a5ca7891904568918c83e761b1038b)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:20:20.311443 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eaadd4f-157f-4eb5-8acd-ab750d32c931 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:20:20.313116 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 579/1160] 10.222.0.49 () {56 vars in 1186 bytes} [Wed Jun 15 10:20:20 2022] GET /identity/v3/projects/b1a5ca7891904568918c83e761b1038b => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:20:20.319109 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e68762a-d9b7-4da9-89d9-02ed97543388 None admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:20:20.348295 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=100 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:20:20.416083 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e68762a-d9b7-4da9-89d9-02ed97543388 None admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:20:20.448902 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778f8c92-15d5-49f1-9c64-9e79b0238324 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:20:20.449496 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778f8c92-15d5-49f1-9c64-9e79b0238324 None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:20:20.449786 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778f8c92-15d5-49f1-9c64-9e79b0238324 None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:20:20.450067 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778f8c92-15d5-49f1-9c64-9e79b0238324 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:20:20.450886 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-778f8c92-15d5-49f1-9c64-9e79b0238324 None admin] RBAC: Adding query filter params (role.id=8051174e7a7947d5b991ebd793c2e543, scope.project.id=b1a5ca7891904568918c83e761b1038b, user.id=16ebd3f590f9472580cfb0d08522d235) {{(pid=80119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:20:20.451231 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-778f8c92-15d5-49f1-9c64-9e79b0238324 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:20:20.452656 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-778f8c92-15d5-49f1-9c64-9e79b0238324 None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:20:20.458322 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 582/1161] 10.222.0.49 () {56 vars in 1374 bytes} [Wed Jun 15 10:20:20 2022] GET /identity/v3/role_assignments?user.id=16ebd3f590f9472580cfb0d08522d235&scope.project.id=b1a5ca7891904568918c83e761b1038b&role.id=8051174e7a7947d5b991ebd793c2e543 => generated 608 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:20:22.569784 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ccb855-d2e6-4694-9200-80da22deaa56 None None] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:20:22.570353 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ccb855-d2e6-4694-9200-80da22deaa56 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:20:22.570353 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ccb855-d2e6-4694-9200-80da22deaa56 None None] PATH_INFO: `/` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:20:22.571730 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 580/1162] 10.222.0.49 () {54 vars in 926 bytes} [Wed Jun 15 10:20:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:20:22.576866 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1385b2-a47a-4e5d-97ad-8f34ce68de79 None None] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:20:22.577283 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1385b2-a47a-4e5d-97ad-8f34ce68de79 None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:20:22.577428 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1385b2-a47a-4e5d-97ad-8f34ce68de79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:20:22.594222 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.auth.core [None req-9b1385b2-a47a-4e5d-97ad-8f34ce68de79 None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:20:22.595467 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-9b1385b2-a47a-4e5d-97ad-8f34ce68de79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:20:22.661072 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 583/1163] 10.222.0.49 () {58 vars in 1023 bytes} [Wed Jun 15 10:20:22 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 10:20:22.683190 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ccb855-d2e6-4694-9200-80da22deaa56 None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:20:22.765203 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:20:22.781464 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ccb855-d2e6-4694-9200-80da22deaa56 None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:20:22.813842 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954039c4-b71a-4758-b65b-eb2140b873ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:20:22.814485 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954039c4-b71a-4758-b65b-eb2140b873ae None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:20:22.814766 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954039c4-b71a-4758-b65b-eb2140b873ae None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:20:22.815039 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954039c4-b71a-4758-b65b-eb2140b873ae None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:20:22.878408 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954039c4-b71a-4758-b65b-eb2140b873ae None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:20:22.879802 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954039c4-b71a-4758-b65b-eb2140b873ae None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:20:22.959494 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:20:22.975173 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 581/1164] 10.222.0.49 () {58 vars in 1282 bytes} [Wed Jun 15 10:20:22 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 15 10:20:22.980929 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1385b2-a47a-4e5d-97ad-8f34ce68de79 None None] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:20:23.004376 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=100 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:20:23.076760 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1385b2-a47a-4e5d-97ad-8f34ce68de79 None None] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:20:23.108553 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827e06e9-dc78-4e56-b25c-17a68d935f8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:20:23.109341 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827e06e9-dc78-4e56-b25c-17a68d935f8b None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:20:23.109626 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827e06e9-dc78-4e56-b25c-17a68d935f8b None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:20:23.109903 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827e06e9-dc78-4e56-b25c-17a68d935f8b None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:20:23.116931 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-827e06e9-dc78-4e56-b25c-17a68d935f8b None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:20:23.118372 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-827e06e9-dc78-4e56-b25c-17a68d935f8b None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:20:23.132779 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application [None req-827e06e9-dc78-4e56-b25c-17a68d935f8b None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Jun 15 10:20:23.132779 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:20:23.132779 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Jun 15 10:20:23.132779 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Jun 15 10:20:23.132779 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 15 10:20:23.132779 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 15 10:20:23.132779 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 15 10:20:23.132779 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 15 10:20:23.132779 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Jun 15 10:20:23.132779 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application Jun 15 10:20:23.132779 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 15 10:20:23.132779 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application Jun 15 10:20:23.132779 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:20:23.132779 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:20:23.132779 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:20:23.132779 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:20:23.132779 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:20:23.133975 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:20:23.133975 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:20:23.133975 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:20:23.133975 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:20:23.133975 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:20:23.133975 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:20:23.133975 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 10:20:23.133975 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 10:20:23.133975 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 10:20:23.133975 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 10:20:23.133975 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:20:23.133975 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:20:23.133975 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:20:23.133975 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:20:23.133975 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:20:23.133975 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:20:23.135049 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:20:23.135049 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application with Lock( Jun 15 10:20:23.135049 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:20:23.135049 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:20:23.135049 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:20:23.135049 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:20:23.135049 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:20:23.135049 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:20:23.135049 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:20:23.135049 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:20:23.135049 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Jun 15 10:20:23.135049 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 10:20:23.135049 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Jun 15 10:20:23.135049 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application Jun 15 10:20:23.135049 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 584/1165] 10.222.0.49 () {56 vars in 1117 bytes} [Wed Jun 15 10:20:22 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:20:23.139716 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954039c4-b71a-4758-b65b-eb2140b873ae None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:20:23.232795 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954039c4-b71a-4758-b65b-eb2140b873ae None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:20:23.265716 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c89ef3d-8644-490f-9608-7fffa9bef5e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:20:23.266364 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c89ef3d-8644-490f-9608-7fffa9bef5e9 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:20:23.266645 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c89ef3d-8644-490f-9608-7fffa9bef5e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:20:23.266919 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c89ef3d-8644-490f-9608-7fffa9bef5e9 None admin] PATH_INFO: `/v3/domains` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:20:23.267662 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c89ef3d-8644-490f-9608-7fffa9bef5e9 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:20:23.267961 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c89ef3d-8644-490f-9608-7fffa9bef5e9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:20:23.269302 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c89ef3d-8644-490f-9608-7fffa9bef5e9 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:20:23.276162 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 582/1166] 10.222.0.49 () {56 vars in 1115 bytes} [Wed Jun 15 10:20:23 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:20:25.355588 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736056f8-043b-4236-963e-97e52fab3b92 None None] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:20:25.356181 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736056f8-043b-4236-963e-97e52fab3b92 None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:20:25.356181 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736056f8-043b-4236-963e-97e52fab3b92 None None] PATH_INFO: `/` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:20:25.357496 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 585/1167] 10.222.0.49 () {54 vars in 926 bytes} [Wed Jun 15 10:20:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:20:25.362698 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a18b10-5b06-4046-91c6-b7e383c124f5 None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:20:25.363170 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a18b10-5b06-4046-91c6-b7e383c124f5 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:20:25.363274 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a18b10-5b06-4046-91c6-b7e383c124f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:20:25.379559 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-13a18b10-5b06-4046-91c6-b7e383c124f5 None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:20:25.380737 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-13a18b10-5b06-4046-91c6-b7e383c124f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:20:25.426845 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:20:25.447515 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 583/1168] 10.222.0.49 () {58 vars in 1023 bytes} [Wed Jun 15 10:20:25 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 10:20:25.470407 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736056f8-043b-4236-963e-97e52fab3b92 None None] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:20:25.553404 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=100 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:20:25.569856 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736056f8-043b-4236-963e-97e52fab3b92 None None] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:20:25.602400 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46202a66-8e05-4e60-b249-de2d8fdd4864 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:20:25.603050 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46202a66-8e05-4e60-b249-de2d8fdd4864 None admin] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:20:25.603360 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46202a66-8e05-4e60-b249-de2d8fdd4864 None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:20:25.603632 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46202a66-8e05-4e60-b249-de2d8fdd4864 None admin] PATH_INFO: `/v3/domains` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:20:25.604373 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46202a66-8e05-4e60-b249-de2d8fdd4864 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:20:25.605778 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46202a66-8e05-4e60-b249-de2d8fdd4864 None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:20:25.619525 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 586/1169] 10.222.0.49 () {60 vars in 1138 bytes} [Wed Jun 15 10:20:25 2022] POST /identity/v3/domains => generated 267 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 10:20:27.757249 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273dd3ad-0d01-45f0-bc78-fcc0ca25c9dc None None] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:20:27.757807 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273dd3ad-0d01-45f0-bc78-fcc0ca25c9dc None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:20:27.757807 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273dd3ad-0d01-45f0-bc78-fcc0ca25c9dc None None] PATH_INFO: `/` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:20:27.759202 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 584/1170] 10.222.0.49 () {54 vars in 926 bytes} [Wed Jun 15 10:20:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:20:27.764368 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f1b4b3-115f-43ef-a65e-e76480991723 None None] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:20:27.764808 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f1b4b3-115f-43ef-a65e-e76480991723 None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:20:27.764939 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f1b4b3-115f-43ef-a65e-e76480991723 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:20:27.781989 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.auth.core [None req-60f1b4b3-115f-43ef-a65e-e76480991723 None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:20:27.783230 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-60f1b4b3-115f-43ef-a65e-e76480991723 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:20:27.847192 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 587/1171] 10.222.0.49 () {58 vars in 1023 bytes} [Wed Jun 15 10:20:27 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 10:20:27.870736 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273dd3ad-0d01-45f0-bc78-fcc0ca25c9dc None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:20:27.967170 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273dd3ad-0d01-45f0-bc78-fcc0ca25c9dc None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:20:27.998532 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9dc5394-6c61-42b4-8863-fd0f75bdb55e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:20:27.999399 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9dc5394-6c61-42b4-8863-fd0f75bdb55e None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:20:27.999679 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9dc5394-6c61-42b4-8863-fd0f75bdb55e None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:20:27.999947 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9dc5394-6c61-42b4-8863-fd0f75bdb55e None admin] PATH_INFO: `/v3/domains/8e7adc99ec9d4455afedced238938688` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:20:28.010429 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9dc5394-6c61-42b4-8863-fd0f75bdb55e None admin] RBAC: Authorizing `identity:get_domain(domain_id=8e7adc99ec9d4455afedced238938688)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:20:28.011829 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9dc5394-6c61-42b4-8863-fd0f75bdb55e None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:20:28.013416 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 585/1172] 10.222.0.49 () {56 vars in 1183 bytes} [Wed Jun 15 10:20:27 2022] GET /identity/v3/domains/8e7adc99ec9d4455afedced238938688 => generated 267 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:20:28.019054 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f1b4b3-115f-43ef-a65e-e76480991723 None None] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:20:28.090682 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:20:28.115410 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f1b4b3-115f-43ef-a65e-e76480991723 None None] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:20:28.148301 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfdf19ad-580b-4caf-802b-4fd4ac8909ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:20:28.148950 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdf19ad-580b-4caf-802b-4fd4ac8909ab None admin] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:20:28.149233 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdf19ad-580b-4caf-802b-4fd4ac8909ab None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:20:28.149510 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdf19ad-580b-4caf-802b-4fd4ac8909ab None admin] PATH_INFO: `/v3/projects` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:20:28.150215 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfdf19ad-580b-4caf-802b-4fd4ac8909ab None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:20:28.151820 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfdf19ad-580b-4caf-802b-4fd4ac8909ab None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:20:28.175055 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 588/1173] 10.222.0.49 () {60 vars in 1141 bytes} [Wed Jun 15 10:20:28 2022] POST /identity/v3/projects => generated 361 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 10:20:30.266865 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0e2459-d9c1-41d3-9967-f28a9736777c None None] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:20:30.267451 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0e2459-d9c1-41d3-9967-f28a9736777c None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:20:30.267668 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0e2459-d9c1-41d3-9967-f28a9736777c None None] PATH_INFO: `/` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:20:30.269041 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 586/1174] 10.222.0.49 () {54 vars in 926 bytes} [Wed Jun 15 10:20:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:20:30.275331 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602480e0-1aa1-4eb7-809c-c5f40aaebb66 None None] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:20:30.275748 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602480e0-1aa1-4eb7-809c-c5f40aaebb66 None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:20:30.275892 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602480e0-1aa1-4eb7-809c-c5f40aaebb66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:20:30.292118 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.auth.core [None req-602480e0-1aa1-4eb7-809c-c5f40aaebb66 None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:20:30.293299 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-602480e0-1aa1-4eb7-809c-c5f40aaebb66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:20:30.356481 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 589/1175] 10.222.0.49 () {58 vars in 1023 bytes} [Wed Jun 15 10:20:30 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 10:20:30.380427 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0e2459-d9c1-41d3-9967-f28a9736777c None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:20:30.422427 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:20:30.479908 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0e2459-d9c1-41d3-9967-f28a9736777c None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:20:30.512120 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbb1a47-f51c-42da-a75b-085995ce4448 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:20:30.512971 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbb1a47-f51c-42da-a75b-085995ce4448 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:20:30.513256 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbb1a47-f51c-42da-a75b-085995ce4448 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:20:30.513531 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbb1a47-f51c-42da-a75b-085995ce4448 None admin] PATH_INFO: `/v3/domains/8e7adc99ec9d4455afedced238938688` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:20:30.514694 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fbb1a47-f51c-42da-a75b-085995ce4448 None admin] RBAC: Authorizing `identity:get_domain(domain_id=8e7adc99ec9d4455afedced238938688)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:20:30.516150 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fbb1a47-f51c-42da-a75b-085995ce4448 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:20:30.517783 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 587/1176] 10.222.0.49 () {56 vars in 1183 bytes} [Wed Jun 15 10:20:30 2022] GET /identity/v3/domains/8e7adc99ec9d4455afedced238938688 => generated 267 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:20:30.523816 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602480e0-1aa1-4eb7-809c-c5f40aaebb66 None None] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:20:30.577036 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:20:30.618838 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602480e0-1aa1-4eb7-809c-c5f40aaebb66 None None] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:20:30.652164 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4215d6db-07fe-4668-8ea7-cc5aa14de482 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:20:30.652820 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4215d6db-07fe-4668-8ea7-cc5aa14de482 None admin] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:20:30.653105 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4215d6db-07fe-4668-8ea7-cc5aa14de482 None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:20:30.653381 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4215d6db-07fe-4668-8ea7-cc5aa14de482 None admin] PATH_INFO: `/v3/users` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:20:30.654092 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4215d6db-07fe-4668-8ea7-cc5aa14de482 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:20:30.655550 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4215d6db-07fe-4668-8ea7-cc5aa14de482 None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:20:30.674182 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 590/1177] 10.222.0.49 () {60 vars in 1132 bytes} [Wed Jun 15 10:20:30 2022] POST /identity/v3/users => generated 311 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 10:20:32.815638 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30161c6-b0cc-4351-bf41-38f56890eec9 None None] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:20:32.816065 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30161c6-b0cc-4351-bf41-38f56890eec9 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:20:32.816202 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30161c6-b0cc-4351-bf41-38f56890eec9 None None] PATH_INFO: `/` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:20:32.817577 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 588/1178] 10.222.0.49 () {54 vars in 926 bytes} [Wed Jun 15 10:20:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:20:32.823417 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc23d62-15ea-4a9b-aa3f-39b510ebe2d9 None None] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:20:32.823800 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc23d62-15ea-4a9b-aa3f-39b510ebe2d9 None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:20:32.823985 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc23d62-15ea-4a9b-aa3f-39b510ebe2d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:20:32.840346 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.auth.core [None req-2cc23d62-15ea-4a9b-aa3f-39b510ebe2d9 None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:20:32.841544 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-2cc23d62-15ea-4a9b-aa3f-39b510ebe2d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:20:32.905820 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 591/1179] 10.222.0.49 () {58 vars in 1023 bytes} [Wed Jun 15 10:20:32 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 10:20:32.929432 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30161c6-b0cc-4351-bf41-38f56890eec9 None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:20:33.027945 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30161c6-b0cc-4351-bf41-38f56890eec9 None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:20:33.036432 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:20:33.061484 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f863b9ff-a550-4d4e-b4d1-65d8f70cb031 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:20:33.062211 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f863b9ff-a550-4d4e-b4d1-65d8f70cb031 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:20:33.062497 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f863b9ff-a550-4d4e-b4d1-65d8f70cb031 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:20:33.062774 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f863b9ff-a550-4d4e-b4d1-65d8f70cb031 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:20:33.073685 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f863b9ff-a550-4d4e-b4d1-65d8f70cb031 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 10:20:33.081296 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application [None req-f863b9ff-a550-4d4e-b4d1-65d8f70cb031 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 10:20:33.081296 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:20:33.081296 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:20:33.081296 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:20:33.081296 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:20:33.081296 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:20:33.081296 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:20:33.081296 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:20:33.081296 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:20:33.081296 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:20:33.081296 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:20:33.081296 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:20:33.081296 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 10:20:33.081296 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 10:20:33.081296 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 10:20:33.081296 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 10:20:33.082457 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 10:20:33.082457 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 10:20:33.082457 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 10:20:33.082457 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 10:20:33.082457 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:20:33.082457 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:20:33.082457 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:20:33.082457 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:20:33.082457 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:20:33.082457 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:20:33.082457 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:20:33.082457 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application with Lock( Jun 15 10:20:33.082457 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:20:33.082457 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:20:33.082457 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:20:33.082457 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:20:33.083781 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:20:33.083781 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:20:33.083781 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:20:33.083781 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:20:33.083781 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 10:20:33.083781 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 10:20:33.083781 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 10:20:33.083781 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 10:20:33.083781 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 10:20:33.083781 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 10:20:33.083781 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 10:20:33.083781 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Jun 15 10:20:33.083781 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 589/1180] 10.222.0.49 () {56 vars in 1096 bytes} [Wed Jun 15 10:20:32 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 10:20:33.088814 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc23d62-15ea-4a9b-aa3f-39b510ebe2d9 None None] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:20:33.130024 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:20:33.185421 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc23d62-15ea-4a9b-aa3f-39b510ebe2d9 None None] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:20:33.218411 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bed1393-f09d-4319-9d20-ce760241c4bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:20:33.219076 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bed1393-f09d-4319-9d20-ce760241c4bc None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:20:33.219391 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bed1393-f09d-4319-9d20-ce760241c4bc None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:20:33.219668 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bed1393-f09d-4319-9d20-ce760241c4bc None admin] PATH_INFO: `/v3/roles` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:20:33.220407 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bed1393-f09d-4319-9d20-ce760241c4bc None admin] RBAC: Adding query filter params (name=admin) {{(pid=80119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:20:33.220715 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bed1393-f09d-4319-9d20-ce760241c4bc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:20:33.222178 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bed1393-f09d-4319-9d20-ce760241c4bc None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:20:33.229964 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 592/1181] 10.222.0.49 () {56 vars in 1099 bytes} [Wed Jun 15 10:20:33 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:20:33.235747 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f863b9ff-a550-4d4e-b4d1-65d8f70cb031 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:20:33.332155 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f863b9ff-a550-4d4e-b4d1-65d8f70cb031 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:20:33.364653 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c76b709-3557-4b2e-a236-a3876f0b308d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:20:33.365503 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c76b709-3557-4b2e-a236-a3876f0b308d None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:20:33.365785 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c76b709-3557-4b2e-a236-a3876f0b308d None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:20:33.366061 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c76b709-3557-4b2e-a236-a3876f0b308d None admin] PATH_INFO: `/v3/users/0ae5f82a31fd439a99d2e03b938fa74b` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:20:33.378719 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c76b709-3557-4b2e-a236-a3876f0b308d None admin] RBAC: Authorizing `identity:get_user(user_id=0ae5f82a31fd439a99d2e03b938fa74b)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:20:33.380208 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c76b709-3557-4b2e-a236-a3876f0b308d None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:20:33.382377 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 590/1182] 10.222.0.49 () {56 vars in 1177 bytes} [Wed Jun 15 10:20:33 2022] GET /identity/v3/users/0ae5f82a31fd439a99d2e03b938fa74b => generated 311 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:20:33.388001 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bed1393-f09d-4319-9d20-ce760241c4bc None admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:20:33.484555 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bed1393-f09d-4319-9d20-ce760241c4bc None admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:20:33.486796 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=100 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:20:33.516816 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19264b18-daf1-413f-94f5-b2271f90a056 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:20:33.517452 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19264b18-daf1-413f-94f5-b2271f90a056 None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:20:33.517732 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19264b18-daf1-413f-94f5-b2271f90a056 None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:20:33.518012 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19264b18-daf1-413f-94f5-b2271f90a056 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:20:33.589133 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19264b18-daf1-413f-94f5-b2271f90a056 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:20:33.590498 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19264b18-daf1-413f-94f5-b2271f90a056 None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:20:33.684145 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 593/1183] 10.222.0.49 () {58 vars in 1282 bytes} [Wed Jun 15 10:20:33 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 15 10:20:33.690096 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c76b709-3557-4b2e-a236-a3876f0b308d None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:20:33.716967 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:20:33.790070 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c76b709-3557-4b2e-a236-a3876f0b308d None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:20:33.822291 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fa094f-b159-4370-9d69-f7d6182a8b30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:20:33.823168 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fa094f-b159-4370-9d69-f7d6182a8b30 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:20:33.823451 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fa094f-b159-4370-9d69-f7d6182a8b30 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:20:33.823722 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fa094f-b159-4370-9d69-f7d6182a8b30 None admin] PATH_INFO: `/v3/projects/3aa07db1cfa44a1e8f70ce09b874f4ad` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:20:33.824888 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49fa094f-b159-4370-9d69-f7d6182a8b30 None admin] RBAC: Authorizing `identity:get_project(project_id=3aa07db1cfa44a1e8f70ce09b874f4ad)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:20:33.826304 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49fa094f-b159-4370-9d69-f7d6182a8b30 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:20:33.828108 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 591/1184] 10.222.0.49 () {56 vars in 1186 bytes} [Wed Jun 15 10:20:33 2022] GET /identity/v3/projects/3aa07db1cfa44a1e8f70ce09b874f4ad => generated 361 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:20:33.834145 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19264b18-daf1-413f-94f5-b2271f90a056 None admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:20:33.837699 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=100 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:20:33.929872 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19264b18-daf1-413f-94f5-b2271f90a056 None admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:20:33.962278 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2615e32b-ff83-4713-85f7-054abfd8977e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:20:33.962919 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2615e32b-ff83-4713-85f7-054abfd8977e None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:20:33.963231 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2615e32b-ff83-4713-85f7-054abfd8977e None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:20:33.963505 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2615e32b-ff83-4713-85f7-054abfd8977e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:20:33.964314 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2615e32b-ff83-4713-85f7-054abfd8977e None admin] RBAC: Adding query filter params (role.id=8051174e7a7947d5b991ebd793c2e543, scope.project.id=3aa07db1cfa44a1e8f70ce09b874f4ad, user.id=0ae5f82a31fd439a99d2e03b938fa74b) {{(pid=80119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:20:33.964620 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2615e32b-ff83-4713-85f7-054abfd8977e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:20:33.966034 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2615e32b-ff83-4713-85f7-054abfd8977e None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:20:33.971276 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 594/1185] 10.222.0.49 () {56 vars in 1374 bytes} [Wed Jun 15 10:20:33 2022] GET /identity/v3/role_assignments?user.id=0ae5f82a31fd439a99d2e03b938fa74b&scope.project.id=3aa07db1cfa44a1e8f70ce09b874f4ad&role.id=8051174e7a7947d5b991ebd793c2e543 => generated 259 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:20:36.043166 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab518551-cb57-401e-a195-00688ce33d5d None None] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:20:36.043665 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab518551-cb57-401e-a195-00688ce33d5d None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:20:36.043665 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab518551-cb57-401e-a195-00688ce33d5d None None] PATH_INFO: `/` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:20:36.045010 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 592/1186] 10.222.0.49 () {54 vars in 926 bytes} [Wed Jun 15 10:20:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:20:36.050328 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59c8347-0cbe-47e9-bbfe-646f09414c1f None None] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:20:36.050752 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59c8347-0cbe-47e9-bbfe-646f09414c1f None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:20:36.050880 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59c8347-0cbe-47e9-bbfe-646f09414c1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:20:36.068281 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.auth.core [None req-c59c8347-0cbe-47e9-bbfe-646f09414c1f None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:20:36.070095 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-c59c8347-0cbe-47e9-bbfe-646f09414c1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:20:36.127768 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=100 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:20:36.133853 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 595/1187] 10.222.0.49 () {58 vars in 1023 bytes} [Wed Jun 15 10:20:36 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 10:20:36.156101 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab518551-cb57-401e-a195-00688ce33d5d None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:20:36.246575 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:20:36.254379 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab518551-cb57-401e-a195-00688ce33d5d None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:20:36.286016 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809924db-ad44-4c30-add0-4519ba58cbed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:20:36.286785 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809924db-ad44-4c30-add0-4519ba58cbed None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:20:36.287065 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809924db-ad44-4c30-add0-4519ba58cbed None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:20:36.287364 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809924db-ad44-4c30-add0-4519ba58cbed None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:20:36.297540 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809924db-ad44-4c30-add0-4519ba58cbed None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 10:20:36.305977 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application [None req-809924db-ad44-4c30-add0-4519ba58cbed None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 10:20:36.305977 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:20:36.305977 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:20:36.305977 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:20:36.305977 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:20:36.305977 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:20:36.305977 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:20:36.305977 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:20:36.305977 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:20:36.305977 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:20:36.305977 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:20:36.305977 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:20:36.305977 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 10:20:36.305977 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 10:20:36.305977 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 10:20:36.305977 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 10:20:36.307075 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 10:20:36.307075 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 10:20:36.307075 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 10:20:36.307075 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 10:20:36.307075 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:20:36.307075 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:20:36.307075 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:20:36.307075 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:20:36.307075 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:20:36.307075 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:20:36.307075 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:20:36.307075 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application with Lock( Jun 15 10:20:36.307075 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:20:36.307075 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:20:36.307075 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:20:36.307075 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:20:36.307997 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:20:36.307997 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:20:36.307997 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:20:36.307997 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:20:36.307997 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 10:20:36.307997 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 10:20:36.307997 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 10:20:36.307997 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 10:20:36.307997 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 10:20:36.307997 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 10:20:36.307997 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 10:20:36.307997 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Jun 15 10:20:36.307997 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 593/1188] 10.222.0.49 () {56 vars in 1096 bytes} [Wed Jun 15 10:20:36 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:20:36.313091 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59c8347-0cbe-47e9-bbfe-646f09414c1f None None] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:20:36.406126 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59c8347-0cbe-47e9-bbfe-646f09414c1f None None] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:20:36.438794 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dee9291-bbe2-43c1-88a9-fb81e0e159fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:20:36.439482 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dee9291-bbe2-43c1-88a9-fb81e0e159fb None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:20:36.439765 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dee9291-bbe2-43c1-88a9-fb81e0e159fb None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:20:36.440038 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dee9291-bbe2-43c1-88a9-fb81e0e159fb None admin] PATH_INFO: `/v3/roles` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:20:36.440770 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dee9291-bbe2-43c1-88a9-fb81e0e159fb None admin] RBAC: Adding query filter params (name=admin) {{(pid=80119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:20:36.441074 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dee9291-bbe2-43c1-88a9-fb81e0e159fb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:20:36.442433 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dee9291-bbe2-43c1-88a9-fb81e0e159fb None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:20:36.450074 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 596/1189] 10.222.0.49 () {56 vars in 1099 bytes} [Wed Jun 15 10:20:36 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:20:36.455654 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809924db-ad44-4c30-add0-4519ba58cbed None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:20:36.552873 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809924db-ad44-4c30-add0-4519ba58cbed None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:20:36.584224 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be9c562-ebf9-4e49-8205-726c0f79c85a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:20:36.585576 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be9c562-ebf9-4e49-8205-726c0f79c85a None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:20:36.585861 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be9c562-ebf9-4e49-8205-726c0f79c85a None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:20:36.586136 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be9c562-ebf9-4e49-8205-726c0f79c85a None admin] PATH_INFO: `/v3/users/0ae5f82a31fd439a99d2e03b938fa74b` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:20:36.587334 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be9c562-ebf9-4e49-8205-726c0f79c85a None admin] RBAC: Authorizing `identity:get_user(user_id=0ae5f82a31fd439a99d2e03b938fa74b)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:20:36.588757 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be9c562-ebf9-4e49-8205-726c0f79c85a None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:20:36.590354 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 594/1190] 10.222.0.49 () {56 vars in 1177 bytes} [Wed Jun 15 10:20:36 2022] GET /identity/v3/users/0ae5f82a31fd439a99d2e03b938fa74b => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:20:36.595858 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dee9291-bbe2-43c1-88a9-fb81e0e159fb None admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:20:36.652187 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=100 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:20:36.691627 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dee9291-bbe2-43c1-88a9-fb81e0e159fb None admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:20:36.723475 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4aaa34-26ce-4544-a6fa-ebcf109038cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:20:36.724314 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4aaa34-26ce-4544-a6fa-ebcf109038cc None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:20:36.724593 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4aaa34-26ce-4544-a6fa-ebcf109038cc None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:20:36.724866 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4aaa34-26ce-4544-a6fa-ebcf109038cc None admin] PATH_INFO: `/v3/projects/3aa07db1cfa44a1e8f70ce09b874f4ad` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:20:36.726557 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e4aaa34-26ce-4544-a6fa-ebcf109038cc None admin] RBAC: Authorizing `identity:get_project(project_id=3aa07db1cfa44a1e8f70ce09b874f4ad)` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:20:36.727994 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e4aaa34-26ce-4544-a6fa-ebcf109038cc None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:20:36.729633 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 597/1191] 10.222.0.49 () {56 vars in 1186 bytes} [Wed Jun 15 10:20:36 2022] GET /identity/v3/projects/3aa07db1cfa44a1e8f70ce09b874f4ad => generated 361 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:20:36.735287 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be9c562-ebf9-4e49-8205-726c0f79c85a None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:20:36.759101 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:20:36.831282 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be9c562-ebf9-4e49-8205-726c0f79c85a None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:20:36.864095 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce3df24-73cc-4f40-96fd-ad77672dae15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:20:36.864976 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce3df24-73cc-4f40-96fd-ad77672dae15 None admin] REQUEST_METHOD: `PUT` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:20:36.865258 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce3df24-73cc-4f40-96fd-ad77672dae15 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:20:36.865528 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce3df24-73cc-4f40-96fd-ad77672dae15 None admin] PATH_INFO: `/v3/projects/3aa07db1cfa44a1e8f70ce09b874f4ad/users/0ae5f82a31fd439a99d2e03b938fa74b/roles/8051174e7a7947d5b991ebd793c2e543` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:20:36.867484 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce3df24-73cc-4f40-96fd-ad77672dae15 None admin] RBAC: Authorizing `identity:create_grant(project_id=3aa07db1cfa44a1e8f70ce09b874f4ad, user_id=0ae5f82a31fd439a99d2e03b938fa74b, role_id=8051174e7a7947d5b991ebd793c2e543)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:20:36.868899 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce3df24-73cc-4f40-96fd-ad77672dae15 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:20:36.881050 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 595/1192] 10.222.0.49 () {58 vars in 1439 bytes} [Wed Jun 15 10:20:36 2022] PUT /identity/v3/projects/3aa07db1cfa44a1e8f70ce09b874f4ad/users/0ae5f82a31fd439a99d2e03b938fa74b/roles/8051174e7a7947d5b991ebd793c2e543 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:20:38.928990 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91edaf1-9ebf-455f-9a52-724ce8e066e9 None None] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:20:38.929505 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91edaf1-9ebf-455f-9a52-724ce8e066e9 None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:20:38.929620 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91edaf1-9ebf-455f-9a52-724ce8e066e9 None None] PATH_INFO: `/` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:20:38.930892 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 598/1193] 10.222.0.49 () {54 vars in 926 bytes} [Wed Jun 15 10:20:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:20:38.935952 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33942639-5d95-43e8-936b-6b0d7a033dfd None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:20:38.936324 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33942639-5d95-43e8-936b-6b0d7a033dfd None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:20:38.936514 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33942639-5d95-43e8-936b-6b0d7a033dfd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:20:38.953432 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-33942639-5d95-43e8-936b-6b0d7a033dfd None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:20:38.954616 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-33942639-5d95-43e8-936b-6b0d7a033dfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:20:39.018474 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 596/1194] 10.222.0.49 () {58 vars in 1023 bytes} [Wed Jun 15 10:20:38 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 10:20:39.040798 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91edaf1-9ebf-455f-9a52-724ce8e066e9 None None] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:20:39.138878 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91edaf1-9ebf-455f-9a52-724ce8e066e9 None None] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:20:39.163754 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=100 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:20:39.171440 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b81109-295d-45f5-8857-cdc9641a259a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:20:39.172225 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b81109-295d-45f5-8857-cdc9641a259a None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:20:39.172506 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b81109-295d-45f5-8857-cdc9641a259a None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:20:39.173028 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b81109-295d-45f5-8857-cdc9641a259a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:20:39.183754 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b81109-295d-45f5-8857-cdc9641a259a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 10:20:39.191263 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application [None req-20b81109-295d-45f5-8857-cdc9641a259a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 10:20:39.191263 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:20:39.191263 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:20:39.191263 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:20:39.191263 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:20:39.191263 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:20:39.191263 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:20:39.191263 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:20:39.191263 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:20:39.191263 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:20:39.191263 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:20:39.191263 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:20:39.191263 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 10:20:39.191263 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 10:20:39.191263 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 10:20:39.191263 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 10:20:39.192406 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 10:20:39.192406 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 10:20:39.192406 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 10:20:39.192406 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 10:20:39.192406 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:20:39.192406 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:20:39.192406 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:20:39.192406 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:20:39.192406 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:20:39.192406 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:20:39.192406 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:20:39.192406 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application with Lock( Jun 15 10:20:39.192406 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:20:39.192406 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:20:39.192406 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:20:39.192406 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:20:39.193464 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:20:39.193464 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:20:39.193464 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:20:39.193464 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:20:39.193464 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 10:20:39.193464 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 10:20:39.193464 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 10:20:39.193464 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 10:20:39.193464 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 10:20:39.193464 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 10:20:39.193464 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 10:20:39.193464 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application Jun 15 10:20:39.193464 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 599/1195] 10.222.0.49 () {56 vars in 1096 bytes} [Wed Jun 15 10:20:39 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:20:39.198301 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33942639-5d95-43e8-936b-6b0d7a033dfd None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:20:39.211957 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:20:39.294089 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33942639-5d95-43e8-936b-6b0d7a033dfd None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:20:39.327268 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ca6066-0c70-4d1f-a3c7-786b70ff122c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:20:39.327933 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ca6066-0c70-4d1f-a3c7-786b70ff122c None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:20:39.328219 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ca6066-0c70-4d1f-a3c7-786b70ff122c None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:20:39.328497 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ca6066-0c70-4d1f-a3c7-786b70ff122c None admin] PATH_INFO: `/v3/roles` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:20:39.329239 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ca6066-0c70-4d1f-a3c7-786b70ff122c None admin] RBAC: Adding query filter params (name=admin) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:20:39.329543 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ca6066-0c70-4d1f-a3c7-786b70ff122c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:20:39.330898 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ca6066-0c70-4d1f-a3c7-786b70ff122c None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:20:39.338713 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 597/1196] 10.222.0.49 () {56 vars in 1099 bytes} [Wed Jun 15 10:20:39 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:20:39.344283 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b81109-295d-45f5-8857-cdc9641a259a None admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:20:39.446829 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b81109-295d-45f5-8857-cdc9641a259a None admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:20:39.479319 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0604ad3f-23cb-4452-8f40-2c768333e1a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:20:39.480181 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0604ad3f-23cb-4452-8f40-2c768333e1a8 None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:20:39.480465 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0604ad3f-23cb-4452-8f40-2c768333e1a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:20:39.480854 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0604ad3f-23cb-4452-8f40-2c768333e1a8 None admin] PATH_INFO: `/v3/users/0ae5f82a31fd439a99d2e03b938fa74b` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:20:39.482706 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0604ad3f-23cb-4452-8f40-2c768333e1a8 None admin] RBAC: Authorizing `identity:get_user(user_id=0ae5f82a31fd439a99d2e03b938fa74b)` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:20:39.485548 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0604ad3f-23cb-4452-8f40-2c768333e1a8 None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:20:39.488609 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 600/1197] 10.222.0.49 () {56 vars in 1177 bytes} [Wed Jun 15 10:20:39 2022] GET /identity/v3/users/0ae5f82a31fd439a99d2e03b938fa74b => generated 311 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:20:39.494856 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ca6066-0c70-4d1f-a3c7-786b70ff122c None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:20:39.565969 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:20:39.590050 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ca6066-0c70-4d1f-a3c7-786b70ff122c None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:20:39.621880 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fcc59a0-874a-4dae-bcb2-7b09eb40519d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:20:39.622508 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fcc59a0-874a-4dae-bcb2-7b09eb40519d None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:20:39.622787 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fcc59a0-874a-4dae-bcb2-7b09eb40519d None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:20:39.623058 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fcc59a0-874a-4dae-bcb2-7b09eb40519d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:20:39.686462 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fcc59a0-874a-4dae-bcb2-7b09eb40519d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:20:39.687911 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fcc59a0-874a-4dae-bcb2-7b09eb40519d None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:20:39.756827 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:20:39.782081 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 598/1198] 10.222.0.49 () {58 vars in 1282 bytes} [Wed Jun 15 10:20:39 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 288 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 15 10:20:39.788466 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0604ad3f-23cb-4452-8f40-2c768333e1a8 None admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:20:39.841971 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=100 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:20:39.883998 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0604ad3f-23cb-4452-8f40-2c768333e1a8 None admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:20:39.917089 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f253c8-fc95-4ac6-9c1b-5831c4fe01d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:20:39.917922 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f253c8-fc95-4ac6-9c1b-5831c4fe01d0 None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:20:39.918201 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f253c8-fc95-4ac6-9c1b-5831c4fe01d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:20:39.918474 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f253c8-fc95-4ac6-9c1b-5831c4fe01d0 None admin] PATH_INFO: `/v3/projects/3aa07db1cfa44a1e8f70ce09b874f4ad` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:20:39.919650 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f253c8-fc95-4ac6-9c1b-5831c4fe01d0 None admin] RBAC: Authorizing `identity:get_project(project_id=3aa07db1cfa44a1e8f70ce09b874f4ad)` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:20:39.921056 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f253c8-fc95-4ac6-9c1b-5831c4fe01d0 None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:20:39.922940 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 601/1199] 10.222.0.49 () {56 vars in 1186 bytes} [Wed Jun 15 10:20:39 2022] GET /identity/v3/projects/3aa07db1cfa44a1e8f70ce09b874f4ad => generated 361 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:20:39.928922 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fcc59a0-874a-4dae-bcb2-7b09eb40519d None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:20:40.023123 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fcc59a0-874a-4dae-bcb2-7b09eb40519d None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:20:40.055212 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd79b04-bdd3-4b9d-8355-bd668e6433ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:20:40.055864 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd79b04-bdd3-4b9d-8355-bd668e6433ae None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:20:40.056151 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd79b04-bdd3-4b9d-8355-bd668e6433ae None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:20:40.056429 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd79b04-bdd3-4b9d-8355-bd668e6433ae None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:20:40.057256 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd79b04-bdd3-4b9d-8355-bd668e6433ae None admin] RBAC: Adding query filter params (role.id=8051174e7a7947d5b991ebd793c2e543, scope.project.id=3aa07db1cfa44a1e8f70ce09b874f4ad, user.id=0ae5f82a31fd439a99d2e03b938fa74b) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:20:40.057569 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd79b04-bdd3-4b9d-8355-bd668e6433ae None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:20:40.058984 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd79b04-bdd3-4b9d-8355-bd668e6433ae None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:20:40.064682 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 599/1200] 10.222.0.49 () {56 vars in 1374 bytes} [Wed Jun 15 10:20:39 2022] GET /identity/v3/role_assignments?user.id=0ae5f82a31fd439a99d2e03b938fa74b&scope.project.id=3aa07db1cfa44a1e8f70ce09b874f4ad&role.id=8051174e7a7947d5b991ebd793c2e543 => generated 608 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:20:49.913081 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=38 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:20:50.062200 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=81 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:21:35.443055 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be7723e-62e0-44d6-addc-c58bfdf934b4 None None] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:21:35.443701 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be7723e-62e0-44d6-addc-c58bfdf934b4 None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:21:35.443701 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be7723e-62e0-44d6-addc-c58bfdf934b4 None None] PATH_INFO: `/` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:21:35.445204 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 602/1201] 10.222.0.49 () {54 vars in 926 bytes} [Wed Jun 15 10:21:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:21:35.450779 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6e6387-ac47-44ac-a3f1-8db5de4db5cc None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:21:35.451319 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6e6387-ac47-44ac-a3f1-8db5de4db5cc None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:21:35.451533 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6e6387-ac47-44ac-a3f1-8db5de4db5cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:21:35.476207 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-ea6e6387-ac47-44ac-a3f1-8db5de4db5cc None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:21:35.477435 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-ea6e6387-ac47-44ac-a3f1-8db5de4db5cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:21:35.558555 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 600/1202] 10.222.0.49 () {58 vars in 1023 bytes} [Wed Jun 15 10:21:35 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 10:21:35.588800 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be7723e-62e0-44d6-addc-c58bfdf934b4 None None] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:21:35.719444 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be7723e-62e0-44d6-addc-c58bfdf934b4 None None] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:21:35.768687 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a79073-d857-4aa4-b5af-9941f3b597a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:21:35.770430 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a79073-d857-4aa4-b5af-9941f3b597a9 None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:21:35.770712 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a79073-d857-4aa4-b5af-9941f3b597a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:21:35.770985 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a79073-d857-4aa4-b5af-9941f3b597a9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:21:35.784219 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a79073-d857-4aa4-b5af-9941f3b597a9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:21:35.786330 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a79073-d857-4aa4-b5af-9941f3b597a9 None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:21:35.809241 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application [None req-32a79073-d857-4aa4-b5af-9941f3b597a9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 10:21:35.809241 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:21:35.809241 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:21:35.809241 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:21:35.809241 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:21:35.809241 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:21:35.809241 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:21:35.809241 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:21:35.809241 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:21:35.809241 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:21:35.809241 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:21:35.809241 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:21:35.809241 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 10:21:35.809241 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 10:21:35.809241 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 10:21:35.810781 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 10:21:35.810781 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:21:35.810781 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:21:35.810781 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:21:35.810781 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:21:35.810781 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:21:35.810781 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:21:35.810781 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:21:35.810781 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application with Lock( Jun 15 10:21:35.810781 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:21:35.810781 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:21:35.810781 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:21:35.810781 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:21:35.810781 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:21:35.810781 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:21:35.810781 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:21:35.810781 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:21:35.812184 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 10:21:35.812184 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 10:21:35.812184 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 10:21:35.812184 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application Jun 15 10:21:35.816461 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 603/1203] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Jun 15 10:21:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:21:35.817020 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6e6387-ac47-44ac-a3f1-8db5de4db5cc None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:21:35.929463 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6e6387-ac47-44ac-a3f1-8db5de4db5cc None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:21:35.968640 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:21:35.975786 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daafb641-a172-4a14-9768-243dfaca4ccb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:21:35.976410 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daafb641-a172-4a14-9768-243dfaca4ccb None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:21:35.976735 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daafb641-a172-4a14-9768-243dfaca4ccb None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:21:35.977075 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daafb641-a172-4a14-9768-243dfaca4ccb None admin] PATH_INFO: `/v3/domains` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:21:35.977833 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daafb641-a172-4a14-9768-243dfaca4ccb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:21:35.978216 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daafb641-a172-4a14-9768-243dfaca4ccb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:21:35.979819 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daafb641-a172-4a14-9768-243dfaca4ccb None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:21:35.996046 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 601/1204] 10.222.0.49 () {56 vars in 1109 bytes} [Wed Jun 15 10:21:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:21:36.001895 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a79073-d857-4aa4-b5af-9941f3b597a9 None admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:21:36.072847 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=100 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:21:36.127869 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a79073-d857-4aa4-b5af-9941f3b597a9 None admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:21:36.176090 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51fda178-a128-4d2d-8f27-df91de5a2ae3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:21:36.176852 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fda178-a128-4d2d-8f27-df91de5a2ae3 None admin] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:21:36.177182 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fda178-a128-4d2d-8f27-df91de5a2ae3 None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:21:36.177524 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fda178-a128-4d2d-8f27-df91de5a2ae3 None admin] PATH_INFO: `/v3/users` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:21:36.178343 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51fda178-a128-4d2d-8f27-df91de5a2ae3 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:21:36.180027 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51fda178-a128-4d2d-8f27-df91de5a2ae3 None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:21:36.209793 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 604/1205] 10.222.0.49 () {60 vars in 1132 bytes} [Wed Jun 15 10:21:36 2022] POST /identity/v3/users => generated 251 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 10:21:38.596875 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf30ac0b-5f5a-4662-bdd1-8496d8810c05 None None] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:21:38.597471 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf30ac0b-5f5a-4662-bdd1-8496d8810c05 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:21:38.597471 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf30ac0b-5f5a-4662-bdd1-8496d8810c05 None None] PATH_INFO: `/` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:21:38.598973 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 602/1206] 10.222.0.49 () {54 vars in 926 bytes} [Wed Jun 15 10:21:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:21:38.604898 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad8f649-81ae-4643-96bf-ed1c122eb0b0 None None] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:21:38.605363 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad8f649-81ae-4643-96bf-ed1c122eb0b0 None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:21:38.605489 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad8f649-81ae-4643-96bf-ed1c122eb0b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:21:38.624041 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.auth.core [None req-0ad8f649-81ae-4643-96bf-ed1c122eb0b0 None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:21:38.625254 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-0ad8f649-81ae-4643-96bf-ed1c122eb0b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:21:38.708263 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 605/1207] 10.222.0.49 () {58 vars in 1023 bytes} [Wed Jun 15 10:21:38 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 15 10:21:38.733114 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf30ac0b-5f5a-4662-bdd1-8496d8810c05 None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:21:38.849168 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf30ac0b-5f5a-4662-bdd1-8496d8810c05 None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:21:38.887499 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1166c94a-dbb1-42fa-8749-ab111e374ba0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:21:38.888406 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1166c94a-dbb1-42fa-8749-ab111e374ba0 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:21:38.888825 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1166c94a-dbb1-42fa-8749-ab111e374ba0 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:21:38.889137 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1166c94a-dbb1-42fa-8749-ab111e374ba0 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:21:38.900351 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1166c94a-dbb1-42fa-8749-ab111e374ba0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 10:21:38.908947 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application [None req-1166c94a-dbb1-42fa-8749-ab111e374ba0 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 15 10:21:38.908947 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:21:38.908947 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:21:38.908947 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:21:38.908947 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:21:38.908947 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:21:38.908947 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:21:38.908947 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:21:38.908947 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:21:38.908947 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:21:38.908947 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:21:38.908947 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:21:38.908947 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 10:21:38.908947 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 10:21:38.908947 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 10:21:38.908947 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 10:21:38.910044 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 10:21:38.910044 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 10:21:38.910044 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 10:21:38.910044 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 10:21:38.910044 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:21:38.910044 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:21:38.910044 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:21:38.910044 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:21:38.910044 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:21:38.910044 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:21:38.910044 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:21:38.910044 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application with Lock( Jun 15 10:21:38.910044 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:21:38.910044 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:21:38.910044 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:21:38.910044 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:21:38.910948 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:21:38.910948 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:21:38.910948 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:21:38.910948 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:21:38.910948 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 10:21:38.910948 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 10:21:38.910948 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 10:21:38.910948 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 10:21:38.910948 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 10:21:38.910948 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 10:21:38.910948 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 15 10:21:38.910948 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Jun 15 10:21:38.910948 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 603/1208] 10.222.0.49 () {56 vars in 1102 bytes} [Wed Jun 15 10:21:38 2022] GET /identity/v3/roles/service => generated 85 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:21:38.916431 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad8f649-81ae-4643-96bf-ed1c122eb0b0 None None] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:21:38.962314 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:21:39.040786 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad8f649-81ae-4643-96bf-ed1c122eb0b0 None None] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:21:39.082561 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0d0feb-c18a-4705-aa60-9fef5926f9b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:21:39.083379 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0d0feb-c18a-4705-aa60-9fef5926f9b8 None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:21:39.083675 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0d0feb-c18a-4705-aa60-9fef5926f9b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:21:39.083959 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0d0feb-c18a-4705-aa60-9fef5926f9b8 None admin] PATH_INFO: `/v3/roles` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:21:39.084721 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a0d0feb-c18a-4705-aa60-9fef5926f9b8 None admin] RBAC: Adding query filter params (name=service) {{(pid=80119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:21:39.085035 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a0d0feb-c18a-4705-aa60-9fef5926f9b8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:21:39.086577 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a0d0feb-c18a-4705-aa60-9fef5926f9b8 None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:21:39.096224 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 606/1209] 10.222.0.49 () {56 vars in 1103 bytes} [Wed Jun 15 10:21:38 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:21:39.103006 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1166c94a-dbb1-42fa-8749-ab111e374ba0 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:21:39.165606 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:21:39.226915 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1166c94a-dbb1-42fa-8749-ab111e374ba0 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:21:39.264844 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eae7011-063e-441f-92ae-a38f8d6cf076 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:21:39.265688 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eae7011-063e-441f-92ae-a38f8d6cf076 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:21:39.265962 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eae7011-063e-441f-92ae-a38f8d6cf076 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:21:39.266237 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eae7011-063e-441f-92ae-a38f8d6cf076 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:21:39.279436 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eae7011-063e-441f-92ae-a38f8d6cf076 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:21:39.281029 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eae7011-063e-441f-92ae-a38f8d6cf076 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:21:39.302779 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application [None req-3eae7011-063e-441f-92ae-a38f8d6cf076 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 10:21:39.302779 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:21:39.302779 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:21:39.302779 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:21:39.302779 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:21:39.302779 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:21:39.302779 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:21:39.302779 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:21:39.302779 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:21:39.302779 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:21:39.302779 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:21:39.302779 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:21:39.302779 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 10:21:39.302779 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 10:21:39.302779 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 10:21:39.303982 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 10:21:39.303982 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:21:39.303982 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:21:39.303982 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:21:39.303982 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:21:39.303982 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:21:39.303982 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:21:39.303982 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:21:39.303982 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application with Lock( Jun 15 10:21:39.303982 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:21:39.303982 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:21:39.303982 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:21:39.303982 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:21:39.303982 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:21:39.303982 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:21:39.303982 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:21:39.303982 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:21:39.309838 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 10:21:39.309838 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 10:21:39.309838 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 10:21:39.309838 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Jun 15 10:21:39.309838 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 604/1210] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Jun 15 10:21:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:21:39.310190 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0d0feb-c18a-4705-aa60-9fef5926f9b8 None admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:21:39.420811 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=100 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:21:39.424365 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0d0feb-c18a-4705-aa60-9fef5926f9b8 None admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:21:39.465218 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9562e24-36b4-4990-a495-138b54fa3fcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:21:39.466249 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9562e24-36b4-4990-a495-138b54fa3fcd None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:21:39.466781 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9562e24-36b4-4990-a495-138b54fa3fcd None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:21:39.467067 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9562e24-36b4-4990-a495-138b54fa3fcd None admin] PATH_INFO: `/v3/domains` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:21:39.467934 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9562e24-36b4-4990-a495-138b54fa3fcd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:21:39.468235 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9562e24-36b4-4990-a495-138b54fa3fcd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:21:39.469722 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9562e24-36b4-4990-a495-138b54fa3fcd None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:21:39.485701 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 607/1211] 10.222.0.49 () {56 vars in 1109 bytes} [Wed Jun 15 10:21:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:21:39.493116 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eae7011-063e-441f-92ae-a38f8d6cf076 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:21:39.621810 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eae7011-063e-441f-92ae-a38f8d6cf076 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:21:39.623654 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:21:39.660869 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81375d72-8862-4d37-9d3f-ce2850d48a9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:21:39.661964 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81375d72-8862-4d37-9d3f-ce2850d48a9c None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:21:39.662250 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81375d72-8862-4d37-9d3f-ce2850d48a9c None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:21:39.662522 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81375d72-8862-4d37-9d3f-ce2850d48a9c None admin] PATH_INFO: `/v3/users/placement` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:21:39.671959 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81375d72-8862-4d37-9d3f-ce2850d48a9c None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:21:39.673608 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81375d72-8862-4d37-9d3f-ce2850d48a9c None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:21:39.689288 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application [None req-81375d72-8862-4d37-9d3f-ce2850d48a9c None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 15 10:21:39.689288 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:21:39.689288 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:21:39.689288 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:21:39.689288 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:21:39.689288 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:21:39.689288 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:21:39.689288 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:21:39.689288 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:21:39.689288 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:21:39.689288 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:21:39.689288 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:21:39.689288 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 15 10:21:39.689288 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 15 10:21:39.689288 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 15 10:21:39.689288 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 15 10:21:39.695171 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:21:39.695171 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:21:39.695171 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 15 10:21:39.695171 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 10:21:39.695171 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 15 10:21:39.695171 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 10:21:39.695171 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:21:39.695171 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:21:39.695171 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:21:39.695171 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:21:39.695171 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:21:39.695171 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application with Lock( Jun 15 10:21:39.695171 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:21:39.695171 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:21:39.695171 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:21:39.695171 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:21:39.695171 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:21:39.695171 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:21:39.696443 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:21:39.696443 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:21:39.696443 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 15 10:21:39.696443 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 15 10:21:39.696443 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 15 10:21:39.696443 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 15 10:21:39.696443 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 15 10:21:39.696443 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 15 10:21:39.696443 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 15 10:21:39.696443 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Jun 15 10:21:39.696443 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 605/1212] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Jun 15 10:21:39 2022] GET /identity/v3/users/placement => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:21:39.699695 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9562e24-36b4-4990-a495-138b54fa3fcd None admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:21:39.825205 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9562e24-36b4-4990-a495-138b54fa3fcd None admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:21:39.871725 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a4f1e7-8468-45f7-a6be-8ed7afbd0701 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:21:39.872473 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a4f1e7-8468-45f7-a6be-8ed7afbd0701 None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:21:39.872681 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a4f1e7-8468-45f7-a6be-8ed7afbd0701 None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:21:39.872956 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a4f1e7-8468-45f7-a6be-8ed7afbd0701 None admin] PATH_INFO: `/v3/users` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:21:39.873876 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a4f1e7-8468-45f7-a6be-8ed7afbd0701 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:21:39.874191 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a4f1e7-8468-45f7-a6be-8ed7afbd0701 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:21:39.877372 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a4f1e7-8468-45f7-a6be-8ed7afbd0701 None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:21:39.899129 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 608/1213] 10.222.0.49 () {56 vars in 1143 bytes} [Wed Jun 15 10:21:39 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:21:39.901315 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81375d72-8862-4d37-9d3f-ce2850d48a9c None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:21:40.023376 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81375d72-8862-4d37-9d3f-ce2850d48a9c None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:21:40.065923 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25dad118-ac23-4011-8ae8-f952fd3a6de4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:21:40.067230 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25dad118-ac23-4011-8ae8-f952fd3a6de4 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:21:40.067561 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25dad118-ac23-4011-8ae8-f952fd3a6de4 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:21:40.067911 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25dad118-ac23-4011-8ae8-f952fd3a6de4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:21:40.105663 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:21:40.151362 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25dad118-ac23-4011-8ae8-f952fd3a6de4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:21:40.152926 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25dad118-ac23-4011-8ae8-f952fd3a6de4 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:21:40.268523 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 606/1214] 10.222.0.49 () {58 vars in 1282 bytes} [Wed Jun 15 10:21:39 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 15 10:21:40.275563 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a4f1e7-8468-45f7-a6be-8ed7afbd0701 None admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:21:40.300095 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=100 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:21:40.403577 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a4f1e7-8468-45f7-a6be-8ed7afbd0701 None admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:21:40.439912 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51c7e2e-aa76-4a81-b3d5-761b947a97f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:21:40.440715 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51c7e2e-aa76-4a81-b3d5-761b947a97f5 None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:21:40.441001 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51c7e2e-aa76-4a81-b3d5-761b947a97f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:21:40.441278 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51c7e2e-aa76-4a81-b3d5-761b947a97f5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:21:40.451879 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e51c7e2e-aa76-4a81-b3d5-761b947a97f5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:21:40.453349 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e51c7e2e-aa76-4a81-b3d5-761b947a97f5 None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:21:40.470851 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application [None req-e51c7e2e-aa76-4a81-b3d5-761b947a97f5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 10:21:40.470851 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:21:40.470851 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:21:40.470851 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:21:40.470851 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:21:40.470851 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:21:40.470851 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:21:40.470851 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:21:40.470851 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:21:40.470851 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:21:40.470851 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:21:40.470851 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:21:40.470851 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 10:21:40.470851 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 10:21:40.470851 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 10:21:40.476858 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 10:21:40.476858 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:21:40.476858 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:21:40.476858 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:21:40.476858 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:21:40.476858 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:21:40.476858 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:21:40.476858 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:21:40.476858 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application with Lock( Jun 15 10:21:40.476858 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:21:40.476858 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:21:40.476858 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:21:40.476858 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:21:40.476858 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:21:40.476858 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:21:40.476858 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:21:40.476858 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:21:40.481874 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 10:21:40.481874 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 10:21:40.481874 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 10:21:40.481874 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application Jun 15 10:21:40.481874 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 609/1215] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Jun 15 10:21:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 10:21:40.482233 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25dad118-ac23-4011-8ae8-f952fd3a6de4 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:21:40.561174 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:21:40.610678 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25dad118-ac23-4011-8ae8-f952fd3a6de4 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:21:40.653770 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d151ce0d-c02c-483e-8715-ba39a3d97c88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:21:40.654807 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d151ce0d-c02c-483e-8715-ba39a3d97c88 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:21:40.655307 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d151ce0d-c02c-483e-8715-ba39a3d97c88 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:21:40.655744 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d151ce0d-c02c-483e-8715-ba39a3d97c88 None admin] PATH_INFO: `/v3/domains` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:21:40.656942 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d151ce0d-c02c-483e-8715-ba39a3d97c88 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:21:40.657423 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d151ce0d-c02c-483e-8715-ba39a3d97c88 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:21:40.659588 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d151ce0d-c02c-483e-8715-ba39a3d97c88 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:21:40.676675 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 607/1216] 10.222.0.49 () {56 vars in 1109 bytes} [Wed Jun 15 10:21:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:21:40.681208 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51c7e2e-aa76-4a81-b3d5-761b947a97f5 None admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:21:40.755661 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=100 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:21:40.795726 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51c7e2e-aa76-4a81-b3d5-761b947a97f5 None admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:21:40.834131 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e336636-7c43-42d0-af35-7fe72db82db1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:21:40.835047 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e336636-7c43-42d0-af35-7fe72db82db1 None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:21:40.835455 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e336636-7c43-42d0-af35-7fe72db82db1 None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:21:40.835830 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e336636-7c43-42d0-af35-7fe72db82db1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:21:40.842799 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e336636-7c43-42d0-af35-7fe72db82db1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:21:40.844413 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e336636-7c43-42d0-af35-7fe72db82db1 None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:21:40.859500 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application [None req-0e336636-7c43-42d0-af35-7fe72db82db1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 10:21:40.859500 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:21:40.859500 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:21:40.859500 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:21:40.859500 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:21:40.859500 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:21:40.859500 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:21:40.859500 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:21:40.859500 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:21:40.859500 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:21:40.859500 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:21:40.859500 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:21:40.859500 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 15 10:21:40.859500 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 15 10:21:40.859500 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 15 10:21:40.860705 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 15 10:21:40.860705 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:21:40.860705 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:21:40.860705 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:21:40.860705 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:21:40.860705 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:21:40.860705 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:21:40.860705 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:21:40.860705 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application with Lock( Jun 15 10:21:40.860705 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:21:40.860705 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:21:40.860705 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:21:40.860705 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:21:40.860705 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:21:40.860705 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:21:40.860705 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:21:40.860705 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:21:40.861846 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 15 10:21:40.861846 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 15 10:21:40.861846 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 15 10:21:40.861846 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 15 10:21:40.861846 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 15 10:21:40.861846 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 15 10:21:40.861846 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 10:21:40.861846 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application Jun 15 10:21:40.864255 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 610/1217] 10.222.0.49 () {56 vars in 1111 bytes} [Wed Jun 15 10:21:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:21:40.870803 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d151ce0d-c02c-483e-8715-ba39a3d97c88 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:21:40.996627 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d151ce0d-c02c-483e-8715-ba39a3d97c88 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:21:41.022854 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:21:41.042048 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8206fd73-5796-4fae-8c79-9a0b693664b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:21:41.042800 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8206fd73-5796-4fae-8c79-9a0b693664b8 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:21:41.043136 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8206fd73-5796-4fae-8c79-9a0b693664b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:21:41.043401 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8206fd73-5796-4fae-8c79-9a0b693664b8 None admin] PATH_INFO: `/v3/projects` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:21:41.044276 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8206fd73-5796-4fae-8c79-9a0b693664b8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:21:41.044656 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8206fd73-5796-4fae-8c79-9a0b693664b8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:21:41.046516 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8206fd73-5796-4fae-8c79-9a0b693664b8 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:21:41.059446 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 608/1218] 10.222.0.49 () {56 vars in 1148 bytes} [Wed Jun 15 10:21:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:21:41.066584 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e336636-7c43-42d0-af35-7fe72db82db1 None admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:21:41.178641 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e336636-7c43-42d0-af35-7fe72db82db1 None admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:21:41.213493 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=100 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:21:41.220852 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d542ed72-0e65-47f8-8153-88ea7978af90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:21:41.221582 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d542ed72-0e65-47f8-8153-88ea7978af90 None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:21:41.221966 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d542ed72-0e65-47f8-8153-88ea7978af90 None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:21:41.222310 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d542ed72-0e65-47f8-8153-88ea7978af90 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:21:41.223850 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d542ed72-0e65-47f8-8153-88ea7978af90 None admin] RBAC: Adding query filter params (role.id=16e76452bb5d4ce68a119ad3c9bde41c, scope.project.id=559b3e713f5844b6ae37151d56dd419f, user.id=dd03612a525644518517069b3fa43522) {{(pid=80119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:21:41.224235 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d542ed72-0e65-47f8-8153-88ea7978af90 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:21:41.225745 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d542ed72-0e65-47f8-8153-88ea7978af90 None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:21:41.231652 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 611/1219] 10.222.0.49 () {56 vars in 1374 bytes} [Wed Jun 15 10:21:41 2022] GET /identity/v3/role_assignments?user.id=dd03612a525644518517069b3fa43522&scope.project.id=559b3e713f5844b6ae37151d56dd419f&role.id=16e76452bb5d4ce68a119ad3c9bde41c => generated 259 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:21:43.777670 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1bfda8-5276-42fb-920d-92d879a0e11f None None] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:21:43.778239 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1bfda8-5276-42fb-920d-92d879a0e11f None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:21:43.778239 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1bfda8-5276-42fb-920d-92d879a0e11f None None] PATH_INFO: `/` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:21:43.779642 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 609/1220] 10.222.0.49 () {54 vars in 926 bytes} [Wed Jun 15 10:21:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:21:43.784892 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06889d39-ff6e-4bb3-8251-ca30e8432a6c None None] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:21:43.785300 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06889d39-ff6e-4bb3-8251-ca30e8432a6c None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:21:43.785450 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06889d39-ff6e-4bb3-8251-ca30e8432a6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:21:43.803760 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.auth.core [None req-06889d39-ff6e-4bb3-8251-ca30e8432a6c None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:21:43.804984 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-06889d39-ff6e-4bb3-8251-ca30e8432a6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:21:43.876087 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 612/1221] 10.222.0.49 () {58 vars in 1023 bytes} [Wed Jun 15 10:21:43 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 10:21:43.905133 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1bfda8-5276-42fb-920d-92d879a0e11f None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:21:44.016361 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1bfda8-5276-42fb-920d-92d879a0e11f None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:21:44.049752 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2ed06d-240e-4417-826a-ba0f99533db3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:21:44.050541 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2ed06d-240e-4417-826a-ba0f99533db3 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:21:44.051372 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2ed06d-240e-4417-826a-ba0f99533db3 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:21:44.051646 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2ed06d-240e-4417-826a-ba0f99533db3 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:21:44.061758 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a2ed06d-240e-4417-826a-ba0f99533db3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 10:21:44.070001 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application [None req-6a2ed06d-240e-4417-826a-ba0f99533db3 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 15 10:21:44.070001 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:21:44.070001 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:21:44.070001 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:21:44.070001 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:21:44.070001 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:21:44.070001 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:21:44.070001 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:21:44.070001 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:21:44.070001 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:21:44.070001 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:21:44.070001 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:21:44.070001 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 10:21:44.070001 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 10:21:44.070001 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 10:21:44.070001 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 10:21:44.071146 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 10:21:44.071146 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 10:21:44.071146 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 10:21:44.071146 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 10:21:44.071146 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:21:44.071146 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:21:44.071146 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:21:44.071146 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:21:44.071146 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:21:44.071146 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:21:44.071146 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:21:44.071146 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application with Lock( Jun 15 10:21:44.071146 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:21:44.071146 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:21:44.071146 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:21:44.071146 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:21:44.072051 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:21:44.072051 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:21:44.072051 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:21:44.072051 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:21:44.072051 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 10:21:44.072051 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 10:21:44.072051 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 10:21:44.072051 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 10:21:44.072051 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 10:21:44.072051 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 10:21:44.072051 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 15 10:21:44.072051 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Jun 15 10:21:44.072051 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 610/1222] 10.222.0.49 () {56 vars in 1102 bytes} [Wed Jun 15 10:21:43 2022] GET /identity/v3/roles/service => generated 85 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:21:44.077523 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06889d39-ff6e-4bb3-8251-ca30e8432a6c None None] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:21:44.179636 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06889d39-ff6e-4bb3-8251-ca30e8432a6c None None] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:21:44.206329 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=100 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:21:44.217322 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5774beab-99f2-448d-8748-be759f39a9b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:21:44.217997 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5774beab-99f2-448d-8748-be759f39a9b6 None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:21:44.218284 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5774beab-99f2-448d-8748-be759f39a9b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:21:44.218565 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5774beab-99f2-448d-8748-be759f39a9b6 None admin] PATH_INFO: `/v3/roles` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:21:44.219391 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5774beab-99f2-448d-8748-be759f39a9b6 None admin] RBAC: Adding query filter params (name=service) {{(pid=80119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:21:44.219702 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5774beab-99f2-448d-8748-be759f39a9b6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:21:44.221080 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5774beab-99f2-448d-8748-be759f39a9b6 None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:21:44.229088 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 613/1223] 10.222.0.49 () {56 vars in 1103 bytes} [Wed Jun 15 10:21:44 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:21:44.234765 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2ed06d-240e-4417-826a-ba0f99533db3 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:21:44.281440 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:21:44.349078 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2ed06d-240e-4417-826a-ba0f99533db3 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:21:44.386297 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a83d734-88a3-4c0d-95e7-7044b3076273 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:21:44.387164 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a83d734-88a3-4c0d-95e7-7044b3076273 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:21:44.387456 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a83d734-88a3-4c0d-95e7-7044b3076273 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:21:44.387742 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a83d734-88a3-4c0d-95e7-7044b3076273 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:21:44.400573 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a83d734-88a3-4c0d-95e7-7044b3076273 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:21:44.402703 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a83d734-88a3-4c0d-95e7-7044b3076273 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:21:44.421328 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application [None req-1a83d734-88a3-4c0d-95e7-7044b3076273 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 10:21:44.421328 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:21:44.421328 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:21:44.421328 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:21:44.421328 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:21:44.421328 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:21:44.421328 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:21:44.421328 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:21:44.421328 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:21:44.421328 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:21:44.421328 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:21:44.421328 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:21:44.421328 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 10:21:44.421328 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 10:21:44.421328 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 10:21:44.422433 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 10:21:44.422433 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:21:44.422433 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:21:44.422433 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:21:44.422433 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:21:44.422433 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:21:44.422433 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:21:44.422433 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:21:44.422433 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application with Lock( Jun 15 10:21:44.422433 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:21:44.422433 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:21:44.422433 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:21:44.422433 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:21:44.422433 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:21:44.422433 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:21:44.422433 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:21:44.422433 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:21:44.423561 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 10:21:44.423561 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 10:21:44.423561 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 10:21:44.423561 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Jun 15 10:21:44.423561 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 611/1224] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Jun 15 10:21:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 10:21:44.429336 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5774beab-99f2-448d-8748-be759f39a9b6 None admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:21:44.547772 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5774beab-99f2-448d-8748-be759f39a9b6 None admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:21:44.587940 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e87863-65ba-45ab-8e6c-860a51c398eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:21:44.588769 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e87863-65ba-45ab-8e6c-860a51c398eb None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:21:44.589140 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e87863-65ba-45ab-8e6c-860a51c398eb None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:21:44.589474 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e87863-65ba-45ab-8e6c-860a51c398eb None admin] PATH_INFO: `/v3/domains` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:21:44.590462 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e87863-65ba-45ab-8e6c-860a51c398eb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:21:44.590782 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e87863-65ba-45ab-8e6c-860a51c398eb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:21:44.592396 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e87863-65ba-45ab-8e6c-860a51c398eb None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:21:44.607456 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 614/1225] 10.222.0.49 () {56 vars in 1109 bytes} [Wed Jun 15 10:21:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 10:21:44.614353 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a83d734-88a3-4c0d-95e7-7044b3076273 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:21:44.712426 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:21:44.719412 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a83d734-88a3-4c0d-95e7-7044b3076273 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:21:44.753470 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71621a8-54aa-4263-ac5d-18738fd67a3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:21:44.754812 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71621a8-54aa-4263-ac5d-18738fd67a3a None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:21:44.755126 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71621a8-54aa-4263-ac5d-18738fd67a3a None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:21:44.755397 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71621a8-54aa-4263-ac5d-18738fd67a3a None admin] PATH_INFO: `/v3/users/placement` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:21:44.762906 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71621a8-54aa-4263-ac5d-18738fd67a3a None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:21:44.764432 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71621a8-54aa-4263-ac5d-18738fd67a3a None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:21:44.778588 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application [None req-b71621a8-54aa-4263-ac5d-18738fd67a3a None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 15 10:21:44.778588 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:21:44.778588 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:21:44.778588 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:21:44.778588 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:21:44.778588 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:21:44.778588 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:21:44.778588 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:21:44.778588 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:21:44.778588 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:21:44.778588 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:21:44.778588 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:21:44.778588 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 15 10:21:44.778588 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 15 10:21:44.778588 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 15 10:21:44.778588 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 15 10:21:44.783732 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:21:44.783732 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:21:44.783732 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 15 10:21:44.783732 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 10:21:44.783732 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 15 10:21:44.783732 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 10:21:44.783732 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:21:44.783732 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:21:44.783732 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:21:44.783732 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:21:44.783732 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:21:44.783732 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application with Lock( Jun 15 10:21:44.783732 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:21:44.783732 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:21:44.783732 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:21:44.783732 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:21:44.783732 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:21:44.783732 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:21:44.785030 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:21:44.785030 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:21:44.785030 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 15 10:21:44.785030 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 15 10:21:44.785030 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 15 10:21:44.785030 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 15 10:21:44.785030 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 15 10:21:44.785030 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 15 10:21:44.785030 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 15 10:21:44.785030 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Jun 15 10:21:44.785030 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 612/1226] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Jun 15 10:21:44 2022] GET /identity/v3/users/placement => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:21:44.786565 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e87863-65ba-45ab-8e6c-860a51c398eb None admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:21:44.838698 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=100 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:21:44.894705 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e87863-65ba-45ab-8e6c-860a51c398eb None admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:21:44.932904 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4824c1-9230-41a4-ae22-ef8616e76f03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:21:44.933581 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4824c1-9230-41a4-ae22-ef8616e76f03 None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:21:44.933866 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4824c1-9230-41a4-ae22-ef8616e76f03 None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:21:44.934201 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4824c1-9230-41a4-ae22-ef8616e76f03 None admin] PATH_INFO: `/v3/users` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:21:44.934982 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f4824c1-9230-41a4-ae22-ef8616e76f03 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:21:44.935330 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f4824c1-9230-41a4-ae22-ef8616e76f03 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:21:44.936769 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f4824c1-9230-41a4-ae22-ef8616e76f03 None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:21:44.960740 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 615/1227] 10.222.0.49 () {56 vars in 1143 bytes} [Wed Jun 15 10:21:44 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:21:44.961952 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71621a8-54aa-4263-ac5d-18738fd67a3a None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:21:45.068871 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71621a8-54aa-4263-ac5d-18738fd67a3a None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:21:45.104759 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15e9d2d-c42c-4b12-bae6-25fcc05e4f53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:21:45.105562 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15e9d2d-c42c-4b12-bae6-25fcc05e4f53 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:21:45.105847 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15e9d2d-c42c-4b12-bae6-25fcc05e4f53 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:21:45.106126 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15e9d2d-c42c-4b12-bae6-25fcc05e4f53 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:21:45.117539 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15e9d2d-c42c-4b12-bae6-25fcc05e4f53 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:21:45.119109 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15e9d2d-c42c-4b12-bae6-25fcc05e4f53 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:21:45.136399 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application [None req-e15e9d2d-c42c-4b12-bae6-25fcc05e4f53 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 10:21:45.136399 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:21:45.136399 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:21:45.136399 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:21:45.136399 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:21:45.136399 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:21:45.136399 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:21:45.136399 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:21:45.136399 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:21:45.136399 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:21:45.136399 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:21:45.136399 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:21:45.136399 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 10:21:45.136399 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 10:21:45.136399 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 10:21:45.137539 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 10:21:45.137539 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:21:45.137539 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:21:45.137539 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:21:45.137539 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:21:45.137539 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:21:45.137539 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:21:45.137539 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:21:45.137539 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application with Lock( Jun 15 10:21:45.137539 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:21:45.137539 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:21:45.137539 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:21:45.137539 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:21:45.137539 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:21:45.137539 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:21:45.137539 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:21:45.137539 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:21:45.138813 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 10:21:45.138813 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 10:21:45.138813 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 10:21:45.138813 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Jun 15 10:21:45.138813 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 613/1228] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Jun 15 10:21:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:21:45.143423 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4824c1-9230-41a4-ae22-ef8616e76f03 None admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:21:45.248284 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4824c1-9230-41a4-ae22-ef8616e76f03 None admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:21:45.251237 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=100 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:21:45.284076 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371eb45f-1c4f-4ab8-9ac8-79709a7d0fcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:21:45.284736 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371eb45f-1c4f-4ab8-9ac8-79709a7d0fcb None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:21:45.285556 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371eb45f-1c4f-4ab8-9ac8-79709a7d0fcb None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:21:45.285833 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371eb45f-1c4f-4ab8-9ac8-79709a7d0fcb None admin] PATH_INFO: `/v3/domains` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:21:45.286555 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-371eb45f-1c4f-4ab8-9ac8-79709a7d0fcb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:21:45.286843 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-371eb45f-1c4f-4ab8-9ac8-79709a7d0fcb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:21:45.288250 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-371eb45f-1c4f-4ab8-9ac8-79709a7d0fcb None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:21:45.305135 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 616/1229] 10.222.0.49 () {56 vars in 1109 bytes} [Wed Jun 15 10:21:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:21:45.306395 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15e9d2d-c42c-4b12-bae6-25fcc05e4f53 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:21:45.317462 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:21:45.416469 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15e9d2d-c42c-4b12-bae6-25fcc05e4f53 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:21:45.451680 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c603217-b1a9-43fd-bccd-06ca7036df41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:21:45.452483 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c603217-b1a9-43fd-bccd-06ca7036df41 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:21:45.452761 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c603217-b1a9-43fd-bccd-06ca7036df41 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:21:45.453032 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c603217-b1a9-43fd-bccd-06ca7036df41 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:21:45.459765 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c603217-b1a9-43fd-bccd-06ca7036df41 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:21:45.461178 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c603217-b1a9-43fd-bccd-06ca7036df41 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:21:45.475816 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application [None req-4c603217-b1a9-43fd-bccd-06ca7036df41 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 10:21:45.475816 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:21:45.475816 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:21:45.475816 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:21:45.475816 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:21:45.475816 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:21:45.475816 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:21:45.475816 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:21:45.475816 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:21:45.475816 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:21:45.475816 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:21:45.475816 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:21:45.475816 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 15 10:21:45.475816 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 15 10:21:45.475816 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 15 10:21:45.476846 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 15 10:21:45.476846 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:21:45.476846 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:21:45.476846 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:21:45.476846 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:21:45.476846 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:21:45.476846 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:21:45.476846 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:21:45.476846 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application with Lock( Jun 15 10:21:45.476846 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:21:45.476846 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:21:45.476846 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:21:45.476846 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:21:45.476846 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:21:45.476846 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:21:45.476846 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:21:45.476846 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:21:45.477991 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 15 10:21:45.477991 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 15 10:21:45.477991 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 15 10:21:45.477991 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 15 10:21:45.477991 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 15 10:21:45.477991 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 15 10:21:45.477991 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 10:21:45.477991 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Jun 15 10:21:45.477991 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 614/1230] 10.222.0.49 () {56 vars in 1111 bytes} [Wed Jun 15 10:21:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:21:45.483295 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371eb45f-1c4f-4ab8-9ac8-79709a7d0fcb None admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:21:45.601707 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371eb45f-1c4f-4ab8-9ac8-79709a7d0fcb None admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:21:45.642291 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aaa29ce-aa59-484a-989e-4ae41f233ead None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:21:45.642958 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aaa29ce-aa59-484a-989e-4ae41f233ead None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:21:45.643278 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aaa29ce-aa59-484a-989e-4ae41f233ead None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:21:45.643557 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aaa29ce-aa59-484a-989e-4ae41f233ead None admin] PATH_INFO: `/v3/projects` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:21:45.644864 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aaa29ce-aa59-484a-989e-4ae41f233ead None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:21:45.645196 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aaa29ce-aa59-484a-989e-4ae41f233ead None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:21:45.646765 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aaa29ce-aa59-484a-989e-4ae41f233ead None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:21:45.661310 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 617/1231] 10.222.0.49 () {56 vars in 1148 bytes} [Wed Jun 15 10:21:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 10:21:45.667640 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c603217-b1a9-43fd-bccd-06ca7036df41 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:21:45.743587 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:21:45.785692 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c603217-b1a9-43fd-bccd-06ca7036df41 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:21:45.824697 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0075e4f-a5e7-45dc-9535-fa534cdc923e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:21:45.825617 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0075e4f-a5e7-45dc-9535-fa534cdc923e None admin] REQUEST_METHOD: `PUT` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:21:45.825898 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0075e4f-a5e7-45dc-9535-fa534cdc923e None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:21:45.826192 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0075e4f-a5e7-45dc-9535-fa534cdc923e None admin] PATH_INFO: `/v3/projects/559b3e713f5844b6ae37151d56dd419f/users/dd03612a525644518517069b3fa43522/roles/16e76452bb5d4ce68a119ad3c9bde41c` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:21:45.840859 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0075e4f-a5e7-45dc-9535-fa534cdc923e None admin] RBAC: Authorizing `identity:create_grant(project_id=559b3e713f5844b6ae37151d56dd419f, user_id=dd03612a525644518517069b3fa43522, role_id=16e76452bb5d4ce68a119ad3c9bde41c)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:21:45.842282 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0075e4f-a5e7-45dc-9535-fa534cdc923e None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:21:45.855857 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 615/1232] 10.222.0.49 () {58 vars in 1439 bytes} [Wed Jun 15 10:21:45 2022] PUT /identity/v3/projects/559b3e713f5844b6ae37151d56dd419f/users/dd03612a525644518517069b3fa43522/roles/16e76452bb5d4ce68a119ad3c9bde41c => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 10:21:48.027153 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d116e33-1319-4af6-898c-57ad9d77c64d None None] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:21:48.027697 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d116e33-1319-4af6-898c-57ad9d77c64d None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:21:48.027798 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d116e33-1319-4af6-898c-57ad9d77c64d None None] PATH_INFO: `/` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:21:48.029102 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 618/1233] 10.222.0.49 () {54 vars in 926 bytes} [Wed Jun 15 10:21:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:21:48.034618 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9a5100-c489-43fe-a5cb-70819ec00016 None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:21:48.035728 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9a5100-c489-43fe-a5cb-70819ec00016 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:21:48.035858 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9a5100-c489-43fe-a5cb-70819ec00016 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:21:48.053313 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-3e9a5100-c489-43fe-a5cb-70819ec00016 None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:21:48.054509 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-3e9a5100-c489-43fe-a5cb-70819ec00016 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:21:48.127194 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 616/1234] 10.222.0.49 () {58 vars in 1023 bytes} [Wed Jun 15 10:21:48 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 10:21:48.153346 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d116e33-1319-4af6-898c-57ad9d77c64d None None] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:21:48.181225 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=100 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:21:48.272188 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d116e33-1319-4af6-898c-57ad9d77c64d None None] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:21:48.312090 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee51335-9a48-4d9b-be63-f435af3caab6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:21:48.313043 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee51335-9a48-4d9b-be63-f435af3caab6 None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:21:48.313325 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee51335-9a48-4d9b-be63-f435af3caab6 None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:21:48.313595 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee51335-9a48-4d9b-be63-f435af3caab6 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:21:48.326212 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee51335-9a48-4d9b-be63-f435af3caab6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 10:21:48.336004 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application [None req-6ee51335-9a48-4d9b-be63-f435af3caab6 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 15 10:21:48.336004 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:21:48.336004 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:21:48.336004 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:21:48.336004 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:21:48.336004 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:21:48.336004 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:21:48.336004 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:21:48.336004 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:21:48.336004 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:21:48.336004 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:21:48.336004 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:21:48.336004 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 10:21:48.336004 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 10:21:48.336004 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 10:21:48.336004 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 10:21:48.336993 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 10:21:48.336993 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 10:21:48.336993 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 10:21:48.336993 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 10:21:48.336993 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:21:48.336993 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:21:48.336993 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:21:48.336993 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:21:48.336993 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:21:48.336993 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:21:48.336993 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:21:48.336993 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application with Lock( Jun 15 10:21:48.336993 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:21:48.336993 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:21:48.336993 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:21:48.336993 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:21:48.337868 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:21:48.337868 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:21:48.337868 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:21:48.337868 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:21:48.337868 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 10:21:48.337868 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 10:21:48.337868 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 10:21:48.337868 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 10:21:48.337868 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 10:21:48.337868 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 10:21:48.337868 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 15 10:21:48.337868 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application Jun 15 10:21:48.337868 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 619/1235] 10.222.0.49 () {56 vars in 1102 bytes} [Wed Jun 15 10:21:48 2022] GET /identity/v3/roles/service => generated 85 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:21:48.343602 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9a5100-c489-43fe-a5cb-70819ec00016 None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:21:48.392844 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:21:48.452028 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9a5100-c489-43fe-a5cb-70819ec00016 None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:21:48.490585 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3940506b-301e-4e05-92a2-cd41c594bd46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:21:48.491313 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3940506b-301e-4e05-92a2-cd41c594bd46 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:21:48.491598 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3940506b-301e-4e05-92a2-cd41c594bd46 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:21:48.491876 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3940506b-301e-4e05-92a2-cd41c594bd46 None admin] PATH_INFO: `/v3/roles` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:21:48.492779 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3940506b-301e-4e05-92a2-cd41c594bd46 None admin] RBAC: Adding query filter params (name=service) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:21:48.493087 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3940506b-301e-4e05-92a2-cd41c594bd46 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:21:48.494612 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3940506b-301e-4e05-92a2-cd41c594bd46 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:21:48.505343 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 617/1236] 10.222.0.49 () {56 vars in 1103 bytes} [Wed Jun 15 10:21:48 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:21:48.511547 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee51335-9a48-4d9b-be63-f435af3caab6 None admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:21:48.608733 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=100 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:21:48.627505 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee51335-9a48-4d9b-be63-f435af3caab6 None admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:21:48.662410 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cccc32e-2f50-470c-b7c0-1ceac3aaef8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:21:48.663236 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cccc32e-2f50-470c-b7c0-1ceac3aaef8f None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:21:48.663521 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cccc32e-2f50-470c-b7c0-1ceac3aaef8f None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:21:48.663796 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cccc32e-2f50-470c-b7c0-1ceac3aaef8f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:21:48.675133 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cccc32e-2f50-470c-b7c0-1ceac3aaef8f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:21:48.677344 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cccc32e-2f50-470c-b7c0-1ceac3aaef8f None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:21:48.696197 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application [None req-5cccc32e-2f50-470c-b7c0-1ceac3aaef8f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 10:21:48.696197 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:21:48.696197 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:21:48.696197 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:21:48.696197 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:21:48.696197 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:21:48.696197 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:21:48.696197 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:21:48.696197 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:21:48.696197 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:21:48.696197 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:21:48.696197 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:21:48.696197 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 10:21:48.696197 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 10:21:48.696197 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 10:21:48.697319 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 10:21:48.697319 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:21:48.697319 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:21:48.697319 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:21:48.697319 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:21:48.697319 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:21:48.697319 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:21:48.697319 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:21:48.697319 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application with Lock( Jun 15 10:21:48.697319 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:21:48.697319 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:21:48.697319 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:21:48.697319 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:21:48.697319 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:21:48.697319 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:21:48.697319 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:21:48.697319 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:21:48.698410 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 10:21:48.698410 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 10:21:48.698410 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 10:21:48.698410 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application Jun 15 10:21:48.698410 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 620/1237] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Jun 15 10:21:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:21:48.703239 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3940506b-301e-4e05-92a2-cd41c594bd46 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:21:48.810745 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3940506b-301e-4e05-92a2-cd41c594bd46 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:21:48.823096 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:21:48.856316 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c129b2d-0543-4639-bfb8-8f3f78b65414 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:21:48.857168 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c129b2d-0543-4639-bfb8-8f3f78b65414 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:21:48.857582 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c129b2d-0543-4639-bfb8-8f3f78b65414 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:21:48.857966 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c129b2d-0543-4639-bfb8-8f3f78b65414 None admin] PATH_INFO: `/v3/domains` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:21:48.858981 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c129b2d-0543-4639-bfb8-8f3f78b65414 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:21:48.859266 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c129b2d-0543-4639-bfb8-8f3f78b65414 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:21:48.860886 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c129b2d-0543-4639-bfb8-8f3f78b65414 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:21:48.875870 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 618/1238] 10.222.0.49 () {56 vars in 1109 bytes} [Wed Jun 15 10:21:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:21:48.883853 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cccc32e-2f50-470c-b7c0-1ceac3aaef8f None admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:21:48.987450 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cccc32e-2f50-470c-b7c0-1ceac3aaef8f None admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:21:49.024562 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8389ac3-f8fd-40c7-983b-9a2c4399008b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:21:49.025589 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8389ac3-f8fd-40c7-983b-9a2c4399008b None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:21:49.025935 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8389ac3-f8fd-40c7-983b-9a2c4399008b None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:21:49.026266 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8389ac3-f8fd-40c7-983b-9a2c4399008b None admin] PATH_INFO: `/v3/users/placement` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:21:49.030243 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=100 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:21:49.036209 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8389ac3-f8fd-40c7-983b-9a2c4399008b None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:21:49.038015 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8389ac3-f8fd-40c7-983b-9a2c4399008b None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:21:49.053393 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application [None req-f8389ac3-f8fd-40c7-983b-9a2c4399008b None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 15 10:21:49.053393 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:21:49.053393 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:21:49.053393 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:21:49.053393 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:21:49.053393 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:21:49.053393 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:21:49.053393 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:21:49.053393 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:21:49.053393 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:21:49.053393 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:21:49.053393 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:21:49.053393 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 15 10:21:49.053393 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 15 10:21:49.053393 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 15 10:21:49.053393 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 15 10:21:49.054607 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:21:49.054607 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:21:49.054607 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 15 10:21:49.054607 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 10:21:49.054607 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 15 10:21:49.054607 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 10:21:49.054607 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:21:49.054607 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:21:49.054607 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:21:49.054607 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:21:49.054607 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:21:49.054607 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application with Lock( Jun 15 10:21:49.054607 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:21:49.054607 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:21:49.054607 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:21:49.054607 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:21:49.054607 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:21:49.054607 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:21:49.055817 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:21:49.055817 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:21:49.055817 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 15 10:21:49.055817 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 15 10:21:49.055817 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 15 10:21:49.055817 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 15 10:21:49.055817 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 15 10:21:49.055817 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 15 10:21:49.055817 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 15 10:21:49.055817 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application Jun 15 10:21:49.055817 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 621/1239] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Jun 15 10:21:48 2022] GET /identity/v3/users/placement => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 10:21:49.060465 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c129b2d-0543-4639-bfb8-8f3f78b65414 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:21:49.167877 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c129b2d-0543-4639-bfb8-8f3f78b65414 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:21:49.203276 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda94e68-9f0b-4949-8145-a4757e7ceb58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:21:49.204145 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda94e68-9f0b-4949-8145-a4757e7ceb58 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:21:49.204431 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda94e68-9f0b-4949-8145-a4757e7ceb58 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:21:49.204715 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda94e68-9f0b-4949-8145-a4757e7ceb58 None admin] PATH_INFO: `/v3/users` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:21:49.205686 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda94e68-9f0b-4949-8145-a4757e7ceb58 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:21:49.206001 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda94e68-9f0b-4949-8145-a4757e7ceb58 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:21:49.207494 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda94e68-9f0b-4949-8145-a4757e7ceb58 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:21:49.226361 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 619/1240] 10.222.0.49 () {56 vars in 1143 bytes} [Wed Jun 15 10:21:49 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:21:49.232229 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8389ac3-f8fd-40c7-983b-9a2c4399008b None admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:21:49.341901 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8389ac3-f8fd-40c7-983b-9a2c4399008b None admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:21:49.380591 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5198f57d-ad76-4b8b-97d0-f705765c92fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:21:49.381373 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5198f57d-ad76-4b8b-97d0-f705765c92fb None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:21:49.381660 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5198f57d-ad76-4b8b-97d0-f705765c92fb None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:21:49.381932 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5198f57d-ad76-4b8b-97d0-f705765c92fb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:21:49.452558 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=100 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:21:49.459996 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5198f57d-ad76-4b8b-97d0-f705765c92fb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:21:49.461517 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5198f57d-ad76-4b8b-97d0-f705765c92fb None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:21:49.576416 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 622/1241] 10.222.0.49 () {58 vars in 1282 bytes} [Wed Jun 15 10:21:49 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 15 10:21:49.584082 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda94e68-9f0b-4949-8145-a4757e7ceb58 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:21:49.615738 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:21:49.691955 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda94e68-9f0b-4949-8145-a4757e7ceb58 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:21:49.729047 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eec4c27-6b90-453e-b49f-f9596f1e56df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:21:49.729763 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eec4c27-6b90-453e-b49f-f9596f1e56df None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:21:49.730043 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eec4c27-6b90-453e-b49f-f9596f1e56df None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:21:49.730314 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eec4c27-6b90-453e-b49f-f9596f1e56df None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:21:49.740905 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eec4c27-6b90-453e-b49f-f9596f1e56df None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:21:49.742361 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eec4c27-6b90-453e-b49f-f9596f1e56df None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:21:49.760776 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application [None req-9eec4c27-6b90-453e-b49f-f9596f1e56df None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 10:21:49.760776 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:21:49.760776 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:21:49.760776 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:21:49.760776 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:21:49.760776 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:21:49.760776 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:21:49.760776 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:21:49.760776 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:21:49.760776 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:21:49.760776 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:21:49.760776 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:21:49.760776 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 10:21:49.760776 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 10:21:49.760776 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 10:21:49.762059 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 10:21:49.762059 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:21:49.762059 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:21:49.762059 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:21:49.762059 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:21:49.762059 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:21:49.762059 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:21:49.762059 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:21:49.762059 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application with Lock( Jun 15 10:21:49.762059 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:21:49.762059 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:21:49.762059 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:21:49.762059 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:21:49.762059 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:21:49.762059 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:21:49.762059 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:21:49.762059 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:21:49.762965 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 10:21:49.762965 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 10:21:49.762965 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 10:21:49.762965 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Jun 15 10:21:49.762965 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 620/1242] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Jun 15 10:21:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:21:49.768001 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5198f57d-ad76-4b8b-97d0-f705765c92fb None admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:21:49.872935 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=100 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:21:49.879987 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5198f57d-ad76-4b8b-97d0-f705765c92fb None admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:21:49.916293 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ed48d5-1e2e-40f0-8c40-178c4fa55e0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:21:49.917077 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ed48d5-1e2e-40f0-8c40-178c4fa55e0f None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:21:49.917369 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ed48d5-1e2e-40f0-8c40-178c4fa55e0f None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:21:49.917646 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ed48d5-1e2e-40f0-8c40-178c4fa55e0f None admin] PATH_INFO: `/v3/domains` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:21:49.918369 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ed48d5-1e2e-40f0-8c40-178c4fa55e0f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:21:49.918676 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ed48d5-1e2e-40f0-8c40-178c4fa55e0f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:21:49.920078 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ed48d5-1e2e-40f0-8c40-178c4fa55e0f None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:21:49.932101 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 623/1243] 10.222.0.49 () {56 vars in 1109 bytes} [Wed Jun 15 10:21:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:21:49.938190 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eec4c27-6b90-453e-b49f-f9596f1e56df None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:21:50.026547 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:21:50.051767 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eec4c27-6b90-453e-b49f-f9596f1e56df None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:21:50.091419 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e7e3a0-0d83-497a-9da5-209febeec82a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:21:50.092635 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e7e3a0-0d83-497a-9da5-209febeec82a None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:21:50.092635 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e7e3a0-0d83-497a-9da5-209febeec82a None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:21:50.092934 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e7e3a0-0d83-497a-9da5-209febeec82a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:21:50.100776 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e7e3a0-0d83-497a-9da5-209febeec82a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:21:50.102236 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e7e3a0-0d83-497a-9da5-209febeec82a None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:21:50.116540 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application [None req-e7e7e3a0-0d83-497a-9da5-209febeec82a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 10:21:50.116540 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:21:50.116540 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:21:50.116540 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:21:50.116540 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:21:50.116540 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:21:50.116540 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:21:50.116540 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:21:50.116540 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:21:50.116540 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:21:50.116540 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:21:50.116540 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:21:50.116540 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 15 10:21:50.116540 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 15 10:21:50.116540 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 15 10:21:50.117768 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 15 10:21:50.117768 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:21:50.117768 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:21:50.117768 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:21:50.117768 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:21:50.117768 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:21:50.117768 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:21:50.117768 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:21:50.117768 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application with Lock( Jun 15 10:21:50.117768 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:21:50.117768 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:21:50.117768 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:21:50.117768 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:21:50.117768 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:21:50.117768 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:21:50.117768 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:21:50.117768 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:21:50.118854 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 15 10:21:50.118854 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 15 10:21:50.118854 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 15 10:21:50.118854 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 15 10:21:50.118854 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 15 10:21:50.118854 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 15 10:21:50.118854 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 10:21:50.118854 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Jun 15 10:21:50.118854 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 621/1244] 10.222.0.49 () {56 vars in 1111 bytes} [Wed Jun 15 10:21:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 10:21:50.123896 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ed48d5-1e2e-40f0-8c40-178c4fa55e0f None admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:21:50.246198 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ed48d5-1e2e-40f0-8c40-178c4fa55e0f None admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:21:50.283509 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b854ae4-be46-4cd7-8ad3-e6a9ab2daaaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:21:50.284180 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b854ae4-be46-4cd7-8ad3-e6a9ab2daaaf None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:21:50.284464 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b854ae4-be46-4cd7-8ad3-e6a9ab2daaaf None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:21:50.284784 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b854ae4-be46-4cd7-8ad3-e6a9ab2daaaf None admin] PATH_INFO: `/v3/projects` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:21:50.286072 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b854ae4-be46-4cd7-8ad3-e6a9ab2daaaf None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:21:50.286378 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b854ae4-be46-4cd7-8ad3-e6a9ab2daaaf None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:21:50.287831 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b854ae4-be46-4cd7-8ad3-e6a9ab2daaaf None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:21:50.302785 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 624/1245] 10.222.0.49 () {56 vars in 1148 bytes} [Wed Jun 15 10:21:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:21:50.309193 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e7e3a0-0d83-497a-9da5-209febeec82a None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:21:50.411626 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e7e3a0-0d83-497a-9da5-209febeec82a None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:21:50.449231 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286cae86-a2dd-4913-8cb7-72d43cd96427 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:21:50.449893 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286cae86-a2dd-4913-8cb7-72d43cd96427 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:21:50.450179 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286cae86-a2dd-4913-8cb7-72d43cd96427 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:21:50.450456 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286cae86-a2dd-4913-8cb7-72d43cd96427 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:21:50.451318 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-286cae86-a2dd-4913-8cb7-72d43cd96427 None admin] RBAC: Adding query filter params (role.id=16e76452bb5d4ce68a119ad3c9bde41c, scope.project.id=559b3e713f5844b6ae37151d56dd419f, user.id=dd03612a525644518517069b3fa43522) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:21:50.451649 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-286cae86-a2dd-4913-8cb7-72d43cd96427 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:21:50.453070 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-286cae86-a2dd-4913-8cb7-72d43cd96427 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:21:50.460085 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 622/1246] 10.222.0.49 () {56 vars in 1374 bytes} [Wed Jun 15 10:21:50 2022] GET /identity/v3/role_assignments?user.id=dd03612a525644518517069b3fa43522&scope.project.id=559b3e713f5844b6ae37151d56dd419f&role.id=16e76452bb5d4ce68a119ad3c9bde41c => generated 608 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:21:52.801975 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b51b3b1-2ee3-44d1-bf92-5cf40daac69e None None] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:21:52.802492 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b51b3b1-2ee3-44d1-bf92-5cf40daac69e None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:21:52.802598 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b51b3b1-2ee3-44d1-bf92-5cf40daac69e None None] PATH_INFO: `/` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:21:52.804091 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 625/1247] 10.222.0.49 () {54 vars in 926 bytes} [Wed Jun 15 10:21:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:21:52.809319 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1c71c0-f9cd-4ad5-92a1-e09814257391 None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:21:52.809754 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1c71c0-f9cd-4ad5-92a1-e09814257391 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:21:52.809883 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1c71c0-f9cd-4ad5-92a1-e09814257391 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:21:52.815392 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:21:52.828672 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-1a1c71c0-f9cd-4ad5-92a1-e09814257391 None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:21:52.829846 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-1a1c71c0-f9cd-4ad5-92a1-e09814257391 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:21:52.904364 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 623/1248] 10.222.0.49 () {58 vars in 1023 bytes} [Wed Jun 15 10:21:52 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 15 10:21:52.929325 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b51b3b1-2ee3-44d1-bf92-5cf40daac69e None None] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:21:52.950787 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=100 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:21:53.052051 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b51b3b1-2ee3-44d1-bf92-5cf40daac69e None None] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:21:53.089256 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705ad8ca-e38c-45fb-bd9f-19cc1f9aef3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:21:53.090382 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705ad8ca-e38c-45fb-bd9f-19cc1f9aef3e None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:21:53.090796 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705ad8ca-e38c-45fb-bd9f-19cc1f9aef3e None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:21:53.091248 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705ad8ca-e38c-45fb-bd9f-19cc1f9aef3e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:21:53.104742 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-705ad8ca-e38c-45fb-bd9f-19cc1f9aef3e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 10:21:53.114381 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application [None req-705ad8ca-e38c-45fb-bd9f-19cc1f9aef3e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 10:21:53.114381 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:21:53.114381 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:21:53.114381 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:21:53.114381 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:21:53.114381 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:21:53.114381 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:21:53.114381 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:21:53.114381 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:21:53.114381 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:21:53.114381 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:21:53.114381 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:21:53.114381 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 10:21:53.114381 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 10:21:53.114381 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 10:21:53.114381 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 10:21:53.115658 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 10:21:53.115658 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 10:21:53.115658 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 10:21:53.115658 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 10:21:53.115658 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:21:53.115658 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:21:53.115658 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:21:53.115658 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:21:53.115658 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:21:53.115658 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:21:53.115658 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:21:53.115658 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application with Lock( Jun 15 10:21:53.115658 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:21:53.115658 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:21:53.115658 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:21:53.115658 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:21:53.116788 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:21:53.116788 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:21:53.116788 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:21:53.116788 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:21:53.116788 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 10:21:53.116788 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 10:21:53.116788 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 10:21:53.116788 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 10:21:53.116788 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 10:21:53.116788 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 10:21:53.116788 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 10:21:53.116788 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application Jun 15 10:21:53.116788 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 626/1249] 10.222.0.49 () {56 vars in 1096 bytes} [Wed Jun 15 10:21:52 2022] GET /identity/v3/roles/admin => generated 83 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:21:53.122286 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1c71c0-f9cd-4ad5-92a1-e09814257391 None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:21:53.232244 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1c71c0-f9cd-4ad5-92a1-e09814257391 None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:21:53.261395 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:21:53.266657 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781f3707-50e4-46b0-a0cb-8a21f484d6e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:21:53.267369 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781f3707-50e4-46b0-a0cb-8a21f484d6e8 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:21:53.267656 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781f3707-50e4-46b0-a0cb-8a21f484d6e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:21:53.267932 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781f3707-50e4-46b0-a0cb-8a21f484d6e8 None admin] PATH_INFO: `/v3/roles` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:21:53.268679 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-781f3707-50e4-46b0-a0cb-8a21f484d6e8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:21:53.269479 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-781f3707-50e4-46b0-a0cb-8a21f484d6e8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:21:53.270855 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-781f3707-50e4-46b0-a0cb-8a21f484d6e8 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:21:53.279625 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 624/1250] 10.222.0.49 () {56 vars in 1099 bytes} [Wed Jun 15 10:21:53 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:21:53.286062 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705ad8ca-e38c-45fb-bd9f-19cc1f9aef3e None admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:21:53.379518 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=100 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:21:53.405490 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705ad8ca-e38c-45fb-bd9f-19cc1f9aef3e None admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:21:53.442635 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421529e7-aef1-456f-94f6-19523d2a7563 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:21:53.443446 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421529e7-aef1-456f-94f6-19523d2a7563 None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:21:53.443728 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421529e7-aef1-456f-94f6-19523d2a7563 None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:21:53.444003 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421529e7-aef1-456f-94f6-19523d2a7563 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:21:53.455539 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-421529e7-aef1-456f-94f6-19523d2a7563 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:21:53.457713 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-421529e7-aef1-456f-94f6-19523d2a7563 None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:21:53.477033 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application [None req-421529e7-aef1-456f-94f6-19523d2a7563 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 10:21:53.477033 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:21:53.477033 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:21:53.477033 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:21:53.477033 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:21:53.477033 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:21:53.477033 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:21:53.477033 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:21:53.477033 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:21:53.477033 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:21:53.477033 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:21:53.477033 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:21:53.477033 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 10:21:53.477033 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 10:21:53.477033 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 10:21:53.478285 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 10:21:53.478285 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:21:53.478285 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:21:53.478285 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:21:53.478285 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:21:53.478285 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:21:53.478285 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:21:53.478285 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:21:53.478285 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application with Lock( Jun 15 10:21:53.478285 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:21:53.478285 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:21:53.478285 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:21:53.478285 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:21:53.478285 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:21:53.478285 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:21:53.478285 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:21:53.478285 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:21:53.479534 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 10:21:53.479534 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 10:21:53.479534 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 10:21:53.479534 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application Jun 15 10:21:53.479534 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 627/1251] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Jun 15 10:21:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:21:53.484603 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781f3707-50e4-46b0-a0cb-8a21f484d6e8 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:21:53.615910 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781f3707-50e4-46b0-a0cb-8a21f484d6e8 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:21:53.657520 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa9eba1-6a82-48ad-91e2-f3ba45960b62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:21:53.658281 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa9eba1-6a82-48ad-91e2-f3ba45960b62 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:21:53.658684 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa9eba1-6a82-48ad-91e2-f3ba45960b62 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:21:53.658965 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa9eba1-6a82-48ad-91e2-f3ba45960b62 None admin] PATH_INFO: `/v3/domains` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:21:53.659871 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa9eba1-6a82-48ad-91e2-f3ba45960b62 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:21:53.660253 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa9eba1-6a82-48ad-91e2-f3ba45960b62 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:21:53.661738 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa9eba1-6a82-48ad-91e2-f3ba45960b62 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:21:53.676453 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 625/1252] 10.222.0.49 () {56 vars in 1109 bytes} [Wed Jun 15 10:21:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:21:53.682529 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421529e7-aef1-456f-94f6-19523d2a7563 None admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:21:53.797878 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421529e7-aef1-456f-94f6-19523d2a7563 None admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:21:53.831606 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=100 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:21:53.840506 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac34a43-d53d-405c-b224-abeb15f47a44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:21:53.841297 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac34a43-d53d-405c-b224-abeb15f47a44 None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:21:53.841580 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac34a43-d53d-405c-b224-abeb15f47a44 None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:21:53.841856 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac34a43-d53d-405c-b224-abeb15f47a44 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:21:53.849633 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac34a43-d53d-405c-b224-abeb15f47a44 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:21:53.851241 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac34a43-d53d-405c-b224-abeb15f47a44 None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:21:53.866015 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application [None req-aac34a43-d53d-405c-b224-abeb15f47a44 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 15 10:21:53.866015 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:21:53.866015 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:21:53.866015 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:21:53.866015 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:21:53.866015 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:21:53.866015 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:21:53.866015 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:21:53.866015 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:21:53.866015 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:21:53.866015 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:21:53.866015 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:21:53.866015 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 15 10:21:53.866015 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 15 10:21:53.866015 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 15 10:21:53.866015 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 15 10:21:53.867113 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:21:53.867113 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:21:53.867113 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 15 10:21:53.867113 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 10:21:53.867113 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 15 10:21:53.867113 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 10:21:53.867113 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:21:53.867113 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:21:53.867113 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:21:53.867113 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:21:53.867113 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:21:53.867113 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application with Lock( Jun 15 10:21:53.867113 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:21:53.867113 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:21:53.867113 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:21:53.867113 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:21:53.867113 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:21:53.867113 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:21:53.868137 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:21:53.868137 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:21:53.868137 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 15 10:21:53.868137 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 15 10:21:53.868137 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 15 10:21:53.868137 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 15 10:21:53.868137 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 15 10:21:53.868137 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 15 10:21:53.868137 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 15 10:21:53.868137 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application Jun 15 10:21:53.868137 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 628/1253] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Jun 15 10:21:53 2022] GET /identity/v3/users/placement => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:21:53.874038 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa9eba1-6a82-48ad-91e2-f3ba45960b62 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:21:53.932387 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:21:53.977989 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa9eba1-6a82-48ad-91e2-f3ba45960b62 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:21:54.014425 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f508c4d-9ef7-42e3-9a55-a4f31de2cf50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:21:54.015122 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f508c4d-9ef7-42e3-9a55-a4f31de2cf50 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:21:54.015432 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f508c4d-9ef7-42e3-9a55-a4f31de2cf50 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:21:54.015684 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f508c4d-9ef7-42e3-9a55-a4f31de2cf50 None admin] PATH_INFO: `/v3/users` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:21:54.016518 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f508c4d-9ef7-42e3-9a55-a4f31de2cf50 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:21:54.016829 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f508c4d-9ef7-42e3-9a55-a4f31de2cf50 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:21:54.018244 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f508c4d-9ef7-42e3-9a55-a4f31de2cf50 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:21:54.035593 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 626/1254] 10.222.0.49 () {56 vars in 1143 bytes} [Wed Jun 15 10:21:53 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 10:21:54.041911 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac34a43-d53d-405c-b224-abeb15f47a44 None admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:21:54.152304 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac34a43-d53d-405c-b224-abeb15f47a44 None admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:21:54.188122 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef83fa0a-de6e-482a-bebb-d354ebc77ebe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:21:54.188716 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef83fa0a-de6e-482a-bebb-d354ebc77ebe None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:21:54.189004 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef83fa0a-de6e-482a-bebb-d354ebc77ebe None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:21:54.189285 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef83fa0a-de6e-482a-bebb-d354ebc77ebe None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:21:54.249637 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=100 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:21:54.262162 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef83fa0a-de6e-482a-bebb-d354ebc77ebe None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:21:54.263678 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef83fa0a-de6e-482a-bebb-d354ebc77ebe None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:21:54.373916 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 629/1255] 10.222.0.49 () {58 vars in 1282 bytes} [Wed Jun 15 10:21:54 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 15 10:21:54.379774 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f508c4d-9ef7-42e3-9a55-a4f31de2cf50 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:21:54.482210 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f508c4d-9ef7-42e3-9a55-a4f31de2cf50 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:21:54.496928 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:21:54.520264 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7fe3dbe-d5a5-47f0-aa3e-3397887c9d04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:21:54.521126 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fe3dbe-d5a5-47f0-aa3e-3397887c9d04 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:21:54.521381 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fe3dbe-d5a5-47f0-aa3e-3397887c9d04 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:21:54.521645 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fe3dbe-d5a5-47f0-aa3e-3397887c9d04 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:21:54.531695 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7fe3dbe-d5a5-47f0-aa3e-3397887c9d04 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:21:54.533164 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7fe3dbe-d5a5-47f0-aa3e-3397887c9d04 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:21:54.552016 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application [None req-f7fe3dbe-d5a5-47f0-aa3e-3397887c9d04 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 10:21:54.552016 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:21:54.552016 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:21:54.552016 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:21:54.552016 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:21:54.552016 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:21:54.552016 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:21:54.552016 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:21:54.552016 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:21:54.552016 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:21:54.552016 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:21:54.552016 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:21:54.552016 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 10:21:54.552016 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 10:21:54.552016 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 10:21:54.553287 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 10:21:54.553287 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:21:54.553287 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:21:54.553287 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:21:54.553287 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:21:54.553287 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:21:54.553287 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:21:54.553287 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:21:54.553287 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application with Lock( Jun 15 10:21:54.553287 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:21:54.553287 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:21:54.553287 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:21:54.553287 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:21:54.553287 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:21:54.553287 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:21:54.553287 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:21:54.553287 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:21:54.554233 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 10:21:54.554233 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 10:21:54.554233 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 10:21:54.554233 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Jun 15 10:21:54.554233 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 627/1256] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Jun 15 10:21:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:21:54.559289 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef83fa0a-de6e-482a-bebb-d354ebc77ebe None admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:21:54.654098 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=100 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:21:54.666885 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef83fa0a-de6e-482a-bebb-d354ebc77ebe None admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:21:54.705729 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89701daf-4873-4387-b6f2-ef71f5205db0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:21:54.706900 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89701daf-4873-4387-b6f2-ef71f5205db0 None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:21:54.707251 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89701daf-4873-4387-b6f2-ef71f5205db0 None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:21:54.707504 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89701daf-4873-4387-b6f2-ef71f5205db0 None admin] PATH_INFO: `/v3/domains` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:21:54.708221 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89701daf-4873-4387-b6f2-ef71f5205db0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:21:54.708528 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89701daf-4873-4387-b6f2-ef71f5205db0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:21:54.709901 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89701daf-4873-4387-b6f2-ef71f5205db0 None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:21:54.721911 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 630/1257] 10.222.0.49 () {56 vars in 1109 bytes} [Wed Jun 15 10:21:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:21:54.727997 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7fe3dbe-d5a5-47f0-aa3e-3397887c9d04 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:21:54.836805 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7fe3dbe-d5a5-47f0-aa3e-3397887c9d04 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:21:54.872554 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ea2ed8-1d4c-439c-96c3-cadf0d054864 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:21:54.873373 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ea2ed8-1d4c-439c-96c3-cadf0d054864 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:21:54.873656 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ea2ed8-1d4c-439c-96c3-cadf0d054864 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:21:54.873931 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ea2ed8-1d4c-439c-96c3-cadf0d054864 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:21:54.882144 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ea2ed8-1d4c-439c-96c3-cadf0d054864 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:21:54.883655 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ea2ed8-1d4c-439c-96c3-cadf0d054864 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:21:54.898780 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application [None req-21ea2ed8-1d4c-439c-96c3-cadf0d054864 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 10:21:54.898780 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:21:54.898780 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:21:54.898780 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:21:54.898780 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:21:54.898780 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:21:54.898780 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:21:54.898780 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:21:54.898780 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:21:54.898780 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:21:54.898780 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:21:54.898780 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:21:54.898780 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 15 10:21:54.898780 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 15 10:21:54.898780 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 15 10:21:54.899832 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 15 10:21:54.899832 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:21:54.899832 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:21:54.899832 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:21:54.899832 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:21:54.899832 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:21:54.899832 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:21:54.899832 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:21:54.899832 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application with Lock( Jun 15 10:21:54.899832 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:21:54.899832 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:21:54.899832 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:21:54.899832 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:21:54.899832 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:21:54.899832 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:21:54.899832 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:21:54.899832 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:21:54.900739 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 15 10:21:54.900739 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 15 10:21:54.900739 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 15 10:21:54.900739 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 15 10:21:54.900739 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 15 10:21:54.900739 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 15 10:21:54.900739 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 10:21:54.900739 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Jun 15 10:21:54.900739 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 628/1258] 10.222.0.49 () {56 vars in 1111 bytes} [Wed Jun 15 10:21:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:21:54.906222 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89701daf-4873-4387-b6f2-ef71f5205db0 None admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:21:55.009823 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89701daf-4873-4387-b6f2-ef71f5205db0 None admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:21:55.044693 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3755e0-37f3-470c-8f00-1f2f791e7419 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:21:55.045273 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3755e0-37f3-470c-8f00-1f2f791e7419 None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:21:55.045555 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3755e0-37f3-470c-8f00-1f2f791e7419 None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:21:55.046080 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3755e0-37f3-470c-8f00-1f2f791e7419 None admin] PATH_INFO: `/v3/projects` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:21:55.046860 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d3755e0-37f3-470c-8f00-1f2f791e7419 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:21:55.047189 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d3755e0-37f3-470c-8f00-1f2f791e7419 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:21:55.048628 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d3755e0-37f3-470c-8f00-1f2f791e7419 None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:21:55.061723 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 631/1259] 10.222.0.49 () {56 vars in 1148 bytes} [Wed Jun 15 10:21:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:21:55.068559 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ea2ed8-1d4c-439c-96c3-cadf0d054864 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:21:55.096029 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:21:55.171621 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ea2ed8-1d4c-439c-96c3-cadf0d054864 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:21:55.204935 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d2650d-cf7b-4e2f-ad63-f798ffa9db8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:21:55.205594 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d2650d-cf7b-4e2f-ad63-f798ffa9db8d None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:21:55.205879 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d2650d-cf7b-4e2f-ad63-f798ffa9db8d None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:21:55.206157 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d2650d-cf7b-4e2f-ad63-f798ffa9db8d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:21:55.207595 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d2650d-cf7b-4e2f-ad63-f798ffa9db8d None admin] RBAC: Adding query filter params (role.id=8051174e7a7947d5b991ebd793c2e543, scope.project.id=559b3e713f5844b6ae37151d56dd419f, user.id=dd03612a525644518517069b3fa43522) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:21:55.207918 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d2650d-cf7b-4e2f-ad63-f798ffa9db8d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:21:55.209410 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d2650d-cf7b-4e2f-ad63-f798ffa9db8d None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:21:55.214962 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 629/1260] 10.222.0.49 () {56 vars in 1374 bytes} [Wed Jun 15 10:21:55 2022] GET /identity/v3/role_assignments?user.id=dd03612a525644518517069b3fa43522&scope.project.id=559b3e713f5844b6ae37151d56dd419f&role.id=8051174e7a7947d5b991ebd793c2e543 => generated 259 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:21:57.299598 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0263914-80be-4ede-8f6b-e12fe338e355 None None] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:21:57.300709 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0263914-80be-4ede-8f6b-e12fe338e355 None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:21:57.300831 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0263914-80be-4ede-8f6b-e12fe338e355 None None] PATH_INFO: `/` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:21:57.302107 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 632/1261] 10.222.0.49 () {54 vars in 926 bytes} [Wed Jun 15 10:21:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:21:57.307336 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a330ac83-d0ff-46d5-9c99-ba1ad4f48dd9 None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:21:57.307738 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a330ac83-d0ff-46d5-9c99-ba1ad4f48dd9 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:21:57.307896 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a330ac83-d0ff-46d5-9c99-ba1ad4f48dd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:21:57.324182 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-a330ac83-d0ff-46d5-9c99-ba1ad4f48dd9 None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:21:57.325371 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-a330ac83-d0ff-46d5-9c99-ba1ad4f48dd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:21:57.389413 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 630/1262] 10.222.0.49 () {58 vars in 1023 bytes} [Wed Jun 15 10:21:57 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 10:21:57.411544 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0263914-80be-4ede-8f6b-e12fe338e355 None None] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:21:57.422333 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=100 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:21:57.508414 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0263914-80be-4ede-8f6b-e12fe338e355 None None] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:21:57.541702 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e033c5-50e9-4af9-b2ac-694a3a83b2e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:21:57.542488 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e033c5-50e9-4af9-b2ac-694a3a83b2e4 None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:21:57.542846 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e033c5-50e9-4af9-b2ac-694a3a83b2e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:21:57.543037 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e033c5-50e9-4af9-b2ac-694a3a83b2e4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:21:57.554197 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e033c5-50e9-4af9-b2ac-694a3a83b2e4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 10:21:57.562277 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application [None req-02e033c5-50e9-4af9-b2ac-694a3a83b2e4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 10:21:57.562277 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:21:57.562277 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:21:57.562277 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:21:57.562277 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:21:57.562277 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:21:57.562277 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:21:57.562277 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:21:57.562277 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:21:57.562277 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:21:57.562277 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:21:57.562277 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:21:57.562277 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 10:21:57.562277 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 10:21:57.562277 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 10:21:57.562277 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 10:21:57.563585 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 10:21:57.563585 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 10:21:57.563585 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 10:21:57.563585 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 10:21:57.563585 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:21:57.563585 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:21:57.563585 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:21:57.563585 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:21:57.563585 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:21:57.563585 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:21:57.563585 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:21:57.563585 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application with Lock( Jun 15 10:21:57.563585 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:21:57.563585 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:21:57.563585 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:21:57.563585 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:21:57.565119 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:21:57.565119 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:21:57.565119 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:21:57.565119 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:21:57.565119 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 10:21:57.565119 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 10:21:57.565119 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 10:21:57.565119 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 10:21:57.565119 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 10:21:57.565119 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 10:21:57.565119 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 10:21:57.565119 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application Jun 15 10:21:57.565119 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 633/1263] 10.222.0.49 () {56 vars in 1096 bytes} [Wed Jun 15 10:21:57 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:21:57.571021 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a330ac83-d0ff-46d5-9c99-ba1ad4f48dd9 None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:21:57.594004 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:21:57.678295 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a330ac83-d0ff-46d5-9c99-ba1ad4f48dd9 None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:21:57.711062 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce00418-cbb1-41b0-92d4-dc86acebb9ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:21:57.711776 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce00418-cbb1-41b0-92d4-dc86acebb9ea None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:21:57.712056 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce00418-cbb1-41b0-92d4-dc86acebb9ea None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:21:57.712329 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce00418-cbb1-41b0-92d4-dc86acebb9ea None admin] PATH_INFO: `/v3/roles` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:21:57.713070 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce00418-cbb1-41b0-92d4-dc86acebb9ea None admin] RBAC: Adding query filter params (name=admin) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:21:57.713372 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce00418-cbb1-41b0-92d4-dc86acebb9ea None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:21:57.714738 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce00418-cbb1-41b0-92d4-dc86acebb9ea None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:21:57.724120 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 631/1264] 10.222.0.49 () {56 vars in 1099 bytes} [Wed Jun 15 10:21:57 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 10:21:57.729662 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e033c5-50e9-4af9-b2ac-694a3a83b2e4 None admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:21:57.799598 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=100 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:21:57.834944 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e033c5-50e9-4af9-b2ac-694a3a83b2e4 None admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:21:57.868877 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f74b3f-27b0-45e8-9936-84ce26458fc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:21:57.869669 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f74b3f-27b0-45e8-9936-84ce26458fc1 None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:21:57.869958 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f74b3f-27b0-45e8-9936-84ce26458fc1 None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:21:57.870233 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f74b3f-27b0-45e8-9936-84ce26458fc1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:21:57.881058 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f74b3f-27b0-45e8-9936-84ce26458fc1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:21:57.882517 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f74b3f-27b0-45e8-9936-84ce26458fc1 None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:21:57.899815 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application [None req-72f74b3f-27b0-45e8-9936-84ce26458fc1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 10:21:57.899815 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:21:57.899815 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:21:57.899815 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:21:57.899815 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:21:57.899815 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:21:57.899815 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:21:57.899815 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:21:57.899815 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:21:57.899815 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:21:57.899815 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:21:57.899815 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:21:57.899815 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 10:21:57.899815 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 10:21:57.899815 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 10:21:57.900878 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 10:21:57.900878 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:21:57.900878 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:21:57.900878 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:21:57.900878 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:21:57.900878 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:21:57.900878 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:21:57.900878 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:21:57.900878 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application with Lock( Jun 15 10:21:57.900878 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:21:57.900878 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:21:57.900878 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:21:57.900878 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:21:57.900878 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:21:57.900878 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:21:57.900878 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:21:57.900878 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:21:57.901801 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 10:21:57.901801 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 10:21:57.901801 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 10:21:57.901801 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application Jun 15 10:21:57.903398 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 634/1265] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Jun 15 10:21:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:21:57.906826 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce00418-cbb1-41b0-92d4-dc86acebb9ea None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:21:57.986996 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:21:58.005840 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce00418-cbb1-41b0-92d4-dc86acebb9ea None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:21:58.037615 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f01c0c-7e4c-4799-b261-24089cdab7ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:21:58.038265 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f01c0c-7e4c-4799-b261-24089cdab7ef None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:21:58.038546 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f01c0c-7e4c-4799-b261-24089cdab7ef None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:21:58.038817 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f01c0c-7e4c-4799-b261-24089cdab7ef None admin] PATH_INFO: `/v3/domains` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:21:58.039556 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f01c0c-7e4c-4799-b261-24089cdab7ef None admin] RBAC: Adding query filter params (name=Default) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:21:58.039860 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f01c0c-7e4c-4799-b261-24089cdab7ef None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:21:58.041212 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f01c0c-7e4c-4799-b261-24089cdab7ef None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:21:58.053150 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 632/1266] 10.222.0.49 () {56 vars in 1109 bytes} [Wed Jun 15 10:21:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:21:58.058857 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f74b3f-27b0-45e8-9936-84ce26458fc1 None admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:21:58.154960 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f74b3f-27b0-45e8-9936-84ce26458fc1 None admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:21:58.174243 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=100 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:21:58.188500 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e60a650-3d28-4939-9aea-20b4aa625007 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:21:58.189294 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e60a650-3d28-4939-9aea-20b4aa625007 None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:21:58.189579 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e60a650-3d28-4939-9aea-20b4aa625007 None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:21:58.189857 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e60a650-3d28-4939-9aea-20b4aa625007 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:21:58.197082 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e60a650-3d28-4939-9aea-20b4aa625007 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:21:58.198530 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e60a650-3d28-4939-9aea-20b4aa625007 None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:21:58.212512 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application [None req-5e60a650-3d28-4939-9aea-20b4aa625007 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 15 10:21:58.212512 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:21:58.212512 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:21:58.212512 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:21:58.212512 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:21:58.212512 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:21:58.212512 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:21:58.212512 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:21:58.212512 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:21:58.212512 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:21:58.212512 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:21:58.212512 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:21:58.212512 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 15 10:21:58.212512 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 15 10:21:58.212512 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 15 10:21:58.212512 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 15 10:21:58.213653 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:21:58.213653 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:21:58.213653 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 15 10:21:58.213653 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 10:21:58.213653 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 15 10:21:58.213653 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 10:21:58.213653 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:21:58.213653 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:21:58.213653 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:21:58.213653 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:21:58.213653 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:21:58.213653 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application with Lock( Jun 15 10:21:58.213653 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:21:58.213653 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:21:58.213653 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:21:58.213653 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:21:58.213653 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:21:58.213653 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:21:58.214825 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:21:58.214825 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:21:58.214825 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 15 10:21:58.214825 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 15 10:21:58.214825 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 15 10:21:58.214825 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 15 10:21:58.214825 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 15 10:21:58.214825 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 15 10:21:58.214825 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 15 10:21:58.214825 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application Jun 15 10:21:58.214825 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 635/1267] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Jun 15 10:21:58 2022] GET /identity/v3/users/placement => generated 87 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:21:58.219321 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f01c0c-7e4c-4799-b261-24089cdab7ef None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:21:58.315741 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f01c0c-7e4c-4799-b261-24089cdab7ef None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:21:58.348760 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d02a1b-039b-4f08-bea0-383a63583dbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:21:58.349411 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d02a1b-039b-4f08-bea0-383a63583dbf None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:21:58.349693 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d02a1b-039b-4f08-bea0-383a63583dbf None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:21:58.349970 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d02a1b-039b-4f08-bea0-383a63583dbf None admin] PATH_INFO: `/v3/users` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:21:58.350792 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d02a1b-039b-4f08-bea0-383a63583dbf None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:21:58.351120 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d02a1b-039b-4f08-bea0-383a63583dbf None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:21:58.352534 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d02a1b-039b-4f08-bea0-383a63583dbf None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:21:58.360750 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:21:58.369027 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 633/1268] 10.222.0.49 () {56 vars in 1143 bytes} [Wed Jun 15 10:21:58 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 10:21:58.374623 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e60a650-3d28-4939-9aea-20b4aa625007 None admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:21:58.471247 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e60a650-3d28-4939-9aea-20b4aa625007 None admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:21:58.503240 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cdd2778-63ad-4b59-952a-1b6696a6ebf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:21:58.504008 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdd2778-63ad-4b59-952a-1b6696a6ebf8 None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:21:58.504293 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdd2778-63ad-4b59-952a-1b6696a6ebf8 None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:21:58.504568 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdd2778-63ad-4b59-952a-1b6696a6ebf8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:21:58.515406 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cdd2778-63ad-4b59-952a-1b6696a6ebf8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:21:58.516838 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cdd2778-63ad-4b59-952a-1b6696a6ebf8 None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:21:58.533707 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application [None req-7cdd2778-63ad-4b59-952a-1b6696a6ebf8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 10:21:58.533707 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:21:58.533707 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:21:58.533707 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:21:58.533707 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:21:58.533707 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:21:58.533707 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:21:58.533707 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:21:58.533707 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:21:58.533707 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:21:58.533707 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:21:58.533707 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:21:58.533707 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 10:21:58.533707 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 10:21:58.533707 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 10:21:58.535407 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 10:21:58.535407 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:21:58.535407 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:21:58.535407 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:21:58.535407 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:21:58.535407 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:21:58.535407 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:21:58.535407 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:21:58.535407 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application with Lock( Jun 15 10:21:58.535407 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:21:58.535407 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:21:58.535407 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:21:58.535407 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:21:58.535407 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:21:58.535407 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:21:58.535407 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:21:58.535407 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:21:58.536529 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 10:21:58.536529 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 10:21:58.536529 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 10:21:58.536529 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application Jun 15 10:21:58.536529 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 636/1269] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Jun 15 10:21:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:21:58.541290 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d02a1b-039b-4f08-bea0-383a63583dbf None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:21:58.646741 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d02a1b-039b-4f08-bea0-383a63583dbf None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:21:58.679038 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42677b24-5b9d-4b21-8d9d-0e4f59e78f53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:21:58.679723 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42677b24-5b9d-4b21-8d9d-0e4f59e78f53 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:21:58.680007 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42677b24-5b9d-4b21-8d9d-0e4f59e78f53 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:21:58.680283 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42677b24-5b9d-4b21-8d9d-0e4f59e78f53 None admin] PATH_INFO: `/v3/domains` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:21:58.680997 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42677b24-5b9d-4b21-8d9d-0e4f59e78f53 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:21:58.681301 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42677b24-5b9d-4b21-8d9d-0e4f59e78f53 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:21:58.682660 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42677b24-5b9d-4b21-8d9d-0e4f59e78f53 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:21:58.693981 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 634/1270] 10.222.0.49 () {56 vars in 1109 bytes} [Wed Jun 15 10:21:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 10:21:58.699650 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cdd2778-63ad-4b59-952a-1b6696a6ebf8 None admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:21:58.730398 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=100 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:21:58.795850 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cdd2778-63ad-4b59-952a-1b6696a6ebf8 None admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:21:58.827924 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d2ef1c-109f-40f9-b678-1a5fdd3b7957 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:21:58.828701 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d2ef1c-109f-40f9-b678-1a5fdd3b7957 None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:21:58.828981 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d2ef1c-109f-40f9-b678-1a5fdd3b7957 None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:21:58.829254 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d2ef1c-109f-40f9-b678-1a5fdd3b7957 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:21:58.836196 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d2ef1c-109f-40f9-b678-1a5fdd3b7957 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:21:58.837622 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d2ef1c-109f-40f9-b678-1a5fdd3b7957 None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:21:58.851355 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application [None req-e7d2ef1c-109f-40f9-b678-1a5fdd3b7957 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 10:21:58.851355 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:21:58.851355 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:21:58.851355 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:21:58.851355 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:21:58.851355 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:21:58.851355 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:21:58.851355 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:21:58.851355 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:21:58.851355 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:21:58.851355 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:21:58.851355 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:21:58.851355 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 15 10:21:58.851355 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 15 10:21:58.851355 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 15 10:21:58.852377 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 15 10:21:58.852377 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:21:58.852377 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:21:58.852377 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:21:58.852377 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:21:58.852377 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:21:58.852377 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:21:58.852377 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:21:58.852377 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application with Lock( Jun 15 10:21:58.852377 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:21:58.852377 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:21:58.852377 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:21:58.852377 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:21:58.852377 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:21:58.852377 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:21:58.852377 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:21:58.852377 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:21:58.853489 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 15 10:21:58.853489 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 15 10:21:58.853489 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 15 10:21:58.853489 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 15 10:21:58.853489 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 15 10:21:58.853489 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 15 10:21:58.853489 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 10:21:58.853489 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application Jun 15 10:21:58.853489 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 637/1271] 10.222.0.49 () {56 vars in 1111 bytes} [Wed Jun 15 10:21:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:21:58.859953 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42677b24-5b9d-4b21-8d9d-0e4f59e78f53 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:21:58.916191 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:21:58.958306 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42677b24-5b9d-4b21-8d9d-0e4f59e78f53 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:21:58.990728 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f755f1e-8a1e-4dfb-8fac-bf05ea039377 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:21:58.991346 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f755f1e-8a1e-4dfb-8fac-bf05ea039377 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:21:58.991630 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f755f1e-8a1e-4dfb-8fac-bf05ea039377 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:21:58.991904 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f755f1e-8a1e-4dfb-8fac-bf05ea039377 None admin] PATH_INFO: `/v3/projects` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:21:58.992681 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f755f1e-8a1e-4dfb-8fac-bf05ea039377 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:21:58.992986 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f755f1e-8a1e-4dfb-8fac-bf05ea039377 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:21:58.994408 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f755f1e-8a1e-4dfb-8fac-bf05ea039377 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:21:59.006591 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 635/1272] 10.222.0.49 () {56 vars in 1148 bytes} [Wed Jun 15 10:21:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:21:59.012372 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d2ef1c-109f-40f9-b678-1a5fdd3b7957 None admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:21:59.098932 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=100 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:21:59.107583 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d2ef1c-109f-40f9-b678-1a5fdd3b7957 None admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:21:59.140321 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d384fd-baf0-43ce-adb4-935549f41061 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:21:59.141188 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d384fd-baf0-43ce-adb4-935549f41061 None admin] REQUEST_METHOD: `PUT` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:21:59.141472 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d384fd-baf0-43ce-adb4-935549f41061 None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:21:59.141747 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d384fd-baf0-43ce-adb4-935549f41061 None admin] PATH_INFO: `/v3/projects/559b3e713f5844b6ae37151d56dd419f/users/dd03612a525644518517069b3fa43522/roles/8051174e7a7947d5b991ebd793c2e543` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:21:59.143668 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d384fd-baf0-43ce-adb4-935549f41061 None admin] RBAC: Authorizing `identity:create_grant(project_id=559b3e713f5844b6ae37151d56dd419f, user_id=dd03612a525644518517069b3fa43522, role_id=8051174e7a7947d5b991ebd793c2e543)` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:21:59.145106 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d384fd-baf0-43ce-adb4-935549f41061 None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:21:59.156952 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 638/1273] 10.222.0.49 () {58 vars in 1439 bytes} [Wed Jun 15 10:21:59 2022] PUT /identity/v3/projects/559b3e713f5844b6ae37151d56dd419f/users/dd03612a525644518517069b3fa43522/roles/8051174e7a7947d5b991ebd793c2e543 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:22:01.217358 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d49f6a4-bfbc-40aa-823c-5f398c1a05d3 None None] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:22:01.217955 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d49f6a4-bfbc-40aa-823c-5f398c1a05d3 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:22:01.217955 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d49f6a4-bfbc-40aa-823c-5f398c1a05d3 None None] PATH_INFO: `/` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:22:01.219225 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 636/1274] 10.222.0.49 () {54 vars in 926 bytes} [Wed Jun 15 10:22:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:22:01.224661 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdfea05-1ab3-46f3-99f0-5cdb77075bd6 None None] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:22:01.225058 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdfea05-1ab3-46f3-99f0-5cdb77075bd6 None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:22:01.225215 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdfea05-1ab3-46f3-99f0-5cdb77075bd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:22:01.242011 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.auth.core [None req-ebdfea05-1ab3-46f3-99f0-5cdb77075bd6 None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:22:01.243229 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-ebdfea05-1ab3-46f3-99f0-5cdb77075bd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:22:01.306985 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 639/1275] 10.222.0.49 () {58 vars in 1023 bytes} [Wed Jun 15 10:22:01 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 10:22:01.329311 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d49f6a4-bfbc-40aa-823c-5f398c1a05d3 None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:22:01.427634 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d49f6a4-bfbc-40aa-823c-5f398c1a05d3 None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:22:01.444192 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:22:01.460732 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a051ec0c-b816-4264-9f92-69674679258f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:22:01.461505 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a051ec0c-b816-4264-9f92-69674679258f None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:22:01.461789 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a051ec0c-b816-4264-9f92-69674679258f None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:22:01.462065 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a051ec0c-b816-4264-9f92-69674679258f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:22:01.472438 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a051ec0c-b816-4264-9f92-69674679258f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 10:22:01.479936 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application [None req-a051ec0c-b816-4264-9f92-69674679258f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 10:22:01.479936 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:22:01.479936 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:22:01.479936 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:22:01.479936 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:22:01.479936 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:22:01.479936 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:22:01.479936 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:22:01.479936 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:22:01.479936 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:22:01.479936 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:22:01.479936 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:22:01.479936 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 10:22:01.479936 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 10:22:01.479936 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 10:22:01.479936 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 10:22:01.481027 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 10:22:01.481027 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 10:22:01.481027 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 10:22:01.481027 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 10:22:01.481027 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:22:01.481027 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:22:01.481027 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:22:01.481027 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:22:01.481027 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:22:01.481027 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:22:01.481027 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:22:01.481027 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application with Lock( Jun 15 10:22:01.481027 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:22:01.481027 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:22:01.481027 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:22:01.481027 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:22:01.482239 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:22:01.482239 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:22:01.482239 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:22:01.482239 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:22:01.482239 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 10:22:01.482239 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 10:22:01.482239 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 10:22:01.482239 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 10:22:01.482239 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 10:22:01.482239 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 10:22:01.482239 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 10:22:01.482239 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Jun 15 10:22:01.482239 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 637/1276] 10.222.0.49 () {56 vars in 1096 bytes} [Wed Jun 15 10:22:01 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:22:01.486839 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebdfea05-1ab3-46f3-99f0-5cdb77075bd6 None None] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:22:01.565686 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:22:01.585584 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebdfea05-1ab3-46f3-99f0-5cdb77075bd6 None None] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:22:01.618237 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada43f4e-69c3-4bf9-a6ac-6f3c37635dee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:22:01.618898 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada43f4e-69c3-4bf9-a6ac-6f3c37635dee None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:22:01.619212 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada43f4e-69c3-4bf9-a6ac-6f3c37635dee None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:22:01.619492 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada43f4e-69c3-4bf9-a6ac-6f3c37635dee None admin] PATH_INFO: `/v3/roles` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:22:01.620231 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada43f4e-69c3-4bf9-a6ac-6f3c37635dee None admin] RBAC: Adding query filter params (name=admin) {{(pid=80119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:22:01.620542 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada43f4e-69c3-4bf9-a6ac-6f3c37635dee None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:22:01.621903 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada43f4e-69c3-4bf9-a6ac-6f3c37635dee None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:22:01.630178 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 640/1277] 10.222.0.49 () {56 vars in 1099 bytes} [Wed Jun 15 10:22:01 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:22:01.635856 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a051ec0c-b816-4264-9f92-69674679258f None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:22:01.731954 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a051ec0c-b816-4264-9f92-69674679258f None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:22:01.763628 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f374ace-2c5c-4819-83b3-0583ba86d763 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:22:01.764396 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f374ace-2c5c-4819-83b3-0583ba86d763 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:22:01.764678 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f374ace-2c5c-4819-83b3-0583ba86d763 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:22:01.764953 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f374ace-2c5c-4819-83b3-0583ba86d763 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:22:01.775869 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f374ace-2c5c-4819-83b3-0583ba86d763 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:22:01.777307 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f374ace-2c5c-4819-83b3-0583ba86d763 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:22:01.793887 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application [None req-3f374ace-2c5c-4819-83b3-0583ba86d763 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 10:22:01.793887 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:22:01.793887 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:22:01.793887 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:22:01.793887 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:22:01.793887 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:22:01.793887 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:22:01.793887 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:22:01.793887 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:22:01.793887 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:22:01.793887 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:22:01.793887 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:22:01.793887 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 10:22:01.793887 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 10:22:01.793887 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 10:22:01.794931 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 10:22:01.794931 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:22:01.794931 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:22:01.794931 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:22:01.794931 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:22:01.794931 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:22:01.794931 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:22:01.794931 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:22:01.794931 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application with Lock( Jun 15 10:22:01.794931 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:22:01.794931 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:22:01.794931 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:22:01.794931 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:22:01.794931 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:22:01.794931 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:22:01.794931 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:22:01.794931 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:22:01.796496 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 10:22:01.796496 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 10:22:01.796496 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 10:22:01.796496 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Jun 15 10:22:01.796496 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 638/1278] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Jun 15 10:22:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:22:01.800984 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada43f4e-69c3-4bf9-a6ac-6f3c37635dee None admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:22:01.897528 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada43f4e-69c3-4bf9-a6ac-6f3c37635dee None admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:22:01.929810 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbbe69a-20b6-4087-9a82-95fa1133abce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:22:01.930454 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbbe69a-20b6-4087-9a82-95fa1133abce None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:22:01.930736 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbbe69a-20b6-4087-9a82-95fa1133abce None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:22:01.931010 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbbe69a-20b6-4087-9a82-95fa1133abce None admin] PATH_INFO: `/v3/domains` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:22:01.931748 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fbbe69a-20b6-4087-9a82-95fa1133abce None admin] RBAC: Adding query filter params (name=Default) {{(pid=80119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:22:01.932052 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fbbe69a-20b6-4087-9a82-95fa1133abce None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:22:01.933406 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fbbe69a-20b6-4087-9a82-95fa1133abce None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:22:01.945756 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 641/1279] 10.222.0.49 () {56 vars in 1109 bytes} [Wed Jun 15 10:22:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:22:01.951326 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f374ace-2c5c-4819-83b3-0583ba86d763 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:22:01.981431 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:22:02.047873 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f374ace-2c5c-4819-83b3-0583ba86d763 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:22:02.080479 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a09cf4d-c9da-4302-a7dc-be374259037d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:22:02.081289 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a09cf4d-c9da-4302-a7dc-be374259037d None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:22:02.081571 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a09cf4d-c9da-4302-a7dc-be374259037d None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:22:02.081842 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a09cf4d-c9da-4302-a7dc-be374259037d None admin] PATH_INFO: `/v3/users/placement` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:22:02.089052 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a09cf4d-c9da-4302-a7dc-be374259037d None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:22:02.090478 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a09cf4d-c9da-4302-a7dc-be374259037d None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:22:02.104243 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application [None req-9a09cf4d-c9da-4302-a7dc-be374259037d None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 15 10:22:02.104243 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:22:02.104243 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:22:02.104243 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:22:02.104243 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:22:02.104243 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:22:02.104243 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:22:02.104243 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:22:02.104243 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:22:02.104243 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:22:02.104243 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:22:02.104243 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:22:02.104243 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 15 10:22:02.104243 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 15 10:22:02.104243 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 15 10:22:02.104243 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 15 10:22:02.105357 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:22:02.105357 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:22:02.105357 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 15 10:22:02.105357 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 10:22:02.105357 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 15 10:22:02.105357 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 10:22:02.105357 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:22:02.105357 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:22:02.105357 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:22:02.105357 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:22:02.105357 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:22:02.105357 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application with Lock( Jun 15 10:22:02.105357 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:22:02.105357 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:22:02.105357 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:22:02.105357 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:22:02.105357 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:22:02.105357 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:22:02.106327 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:22:02.106327 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:22:02.106327 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 15 10:22:02.106327 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 15 10:22:02.106327 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 15 10:22:02.106327 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 15 10:22:02.106327 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 15 10:22:02.106327 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 15 10:22:02.106327 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 15 10:22:02.106327 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Jun 15 10:22:02.106327 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 639/1280] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Jun 15 10:22:01 2022] GET /identity/v3/users/placement => generated 87 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:22:02.111196 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbbe69a-20b6-4087-9a82-95fa1133abce None admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:22:02.115570 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=100 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:22:02.206947 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbbe69a-20b6-4087-9a82-95fa1133abce None admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:22:02.239154 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9a17a9-75e3-49ad-a2a5-c8fa36458de1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:22:02.239814 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9a17a9-75e3-49ad-a2a5-c8fa36458de1 None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:22:02.240095 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9a17a9-75e3-49ad-a2a5-c8fa36458de1 None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:22:02.240371 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9a17a9-75e3-49ad-a2a5-c8fa36458de1 None admin] PATH_INFO: `/v3/users` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:22:02.241189 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a9a17a9-75e3-49ad-a2a5-c8fa36458de1 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:22:02.241496 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a9a17a9-75e3-49ad-a2a5-c8fa36458de1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:22:02.242908 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a9a17a9-75e3-49ad-a2a5-c8fa36458de1 None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:22:02.259677 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 642/1281] 10.222.0.49 () {56 vars in 1143 bytes} [Wed Jun 15 10:22:02 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:22:02.265258 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a09cf4d-c9da-4302-a7dc-be374259037d None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:22:02.351607 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:22:02.361294 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a09cf4d-c9da-4302-a7dc-be374259037d None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:22:02.394067 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9fe3c1f-d631-48e3-8dc2-303f83e9549e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:22:02.394700 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fe3c1f-d631-48e3-8dc2-303f83e9549e None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:22:02.394986 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fe3c1f-d631-48e3-8dc2-303f83e9549e None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:22:02.395296 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fe3c1f-d631-48e3-8dc2-303f83e9549e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:22:02.461309 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9fe3c1f-d631-48e3-8dc2-303f83e9549e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:22:02.462736 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9fe3c1f-d631-48e3-8dc2-303f83e9549e None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:22:02.549284 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:22:02.561092 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 640/1282] 10.222.0.49 () {58 vars in 1282 bytes} [Wed Jun 15 10:22:02 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 15 10:22:02.566919 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9a17a9-75e3-49ad-a2a5-c8fa36458de1 None admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:22:02.625974 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=100 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:22:02.662866 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9a17a9-75e3-49ad-a2a5-c8fa36458de1 None admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:22:02.695550 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ad852c-d19e-4163-bd5d-fe9ae32233ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:22:02.696335 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ad852c-d19e-4163-bd5d-fe9ae32233ff None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:22:02.696616 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ad852c-d19e-4163-bd5d-fe9ae32233ff None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:22:02.696891 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ad852c-d19e-4163-bd5d-fe9ae32233ff None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:22:02.706554 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ad852c-d19e-4163-bd5d-fe9ae32233ff None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:22:02.708024 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ad852c-d19e-4163-bd5d-fe9ae32233ff None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:22:02.725413 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application [None req-f6ad852c-d19e-4163-bd5d-fe9ae32233ff None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 10:22:02.725413 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:22:02.725413 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:22:02.725413 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:22:02.725413 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:22:02.725413 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:22:02.725413 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:22:02.725413 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:22:02.725413 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:22:02.725413 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:22:02.725413 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:22:02.725413 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:22:02.725413 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 10:22:02.725413 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 10:22:02.725413 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 10:22:02.726367 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 10:22:02.726367 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:22:02.726367 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:22:02.726367 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:22:02.726367 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:22:02.726367 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:22:02.726367 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:22:02.726367 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:22:02.726367 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application with Lock( Jun 15 10:22:02.726367 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:22:02.726367 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:22:02.726367 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:22:02.726367 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:22:02.726367 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:22:02.726367 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:22:02.726367 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:22:02.726367 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:22:02.727357 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 10:22:02.727357 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 10:22:02.727357 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 10:22:02.727357 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application Jun 15 10:22:02.727357 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 643/1283] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Jun 15 10:22:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:22:02.732045 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9fe3c1f-d631-48e3-8dc2-303f83e9549e None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:22:02.826577 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9fe3c1f-d631-48e3-8dc2-303f83e9549e None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:22:02.862151 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8df4996-debe-4f57-9cca-d6f1b088dfb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:22:02.862844 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8df4996-debe-4f57-9cca-d6f1b088dfb7 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:22:02.863152 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8df4996-debe-4f57-9cca-d6f1b088dfb7 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:22:02.863427 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8df4996-debe-4f57-9cca-d6f1b088dfb7 None admin] PATH_INFO: `/v3/domains` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:22:02.864136 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8df4996-debe-4f57-9cca-d6f1b088dfb7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:22:02.864437 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8df4996-debe-4f57-9cca-d6f1b088dfb7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:22:02.865790 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8df4996-debe-4f57-9cca-d6f1b088dfb7 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:22:02.877573 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 641/1284] 10.222.0.49 () {56 vars in 1109 bytes} [Wed Jun 15 10:22:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:22:02.882132 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ad852c-d19e-4163-bd5d-fe9ae32233ff None admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:22:02.978136 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ad852c-d19e-4163-bd5d-fe9ae32233ff None admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:22:02.991923 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=100 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:22:03.011791 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf642b84-200a-4fae-b1b4-5ff5af47a489 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:22:03.012565 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf642b84-200a-4fae-b1b4-5ff5af47a489 None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:22:03.012850 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf642b84-200a-4fae-b1b4-5ff5af47a489 None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:22:03.013126 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf642b84-200a-4fae-b1b4-5ff5af47a489 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:22:03.019597 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf642b84-200a-4fae-b1b4-5ff5af47a489 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:22:03.021033 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf642b84-200a-4fae-b1b4-5ff5af47a489 None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:22:03.034725 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application [None req-cf642b84-200a-4fae-b1b4-5ff5af47a489 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 10:22:03.034725 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:22:03.034725 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:22:03.034725 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:22:03.034725 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:22:03.034725 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:22:03.034725 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:22:03.034725 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:22:03.034725 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:22:03.034725 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:22:03.034725 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:22:03.034725 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:22:03.034725 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 15 10:22:03.034725 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 15 10:22:03.034725 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 15 10:22:03.035884 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 15 10:22:03.035884 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:22:03.035884 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:22:03.035884 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:22:03.035884 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:22:03.035884 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:22:03.035884 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:22:03.035884 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:22:03.035884 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application with Lock( Jun 15 10:22:03.035884 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:22:03.035884 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:22:03.035884 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:22:03.035884 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:22:03.035884 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:22:03.035884 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:22:03.035884 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:22:03.035884 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:22:03.037155 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 15 10:22:03.037155 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 15 10:22:03.037155 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 15 10:22:03.037155 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 15 10:22:03.037155 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 15 10:22:03.037155 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 15 10:22:03.037155 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 10:22:03.037155 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: ERROR keystone.server.flask.application Jun 15 10:22:03.037155 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 644/1285] 10.222.0.49 () {56 vars in 1111 bytes} [Wed Jun 15 10:22:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:22:03.041648 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8df4996-debe-4f57-9cca-d6f1b088dfb7 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:22:03.090043 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:22:03.138772 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8df4996-debe-4f57-9cca-d6f1b088dfb7 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:22:03.170943 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed207c9e-d436-464c-8f16-5b233611cc21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:22:03.171621 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed207c9e-d436-464c-8f16-5b233611cc21 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:22:03.171904 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed207c9e-d436-464c-8f16-5b233611cc21 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:22:03.172179 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed207c9e-d436-464c-8f16-5b233611cc21 None admin] PATH_INFO: `/v3/projects` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:22:03.172972 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed207c9e-d436-464c-8f16-5b233611cc21 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:22:03.173275 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed207c9e-d436-464c-8f16-5b233611cc21 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:22:03.174675 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed207c9e-d436-464c-8f16-5b233611cc21 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:22:03.186640 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 642/1286] 10.222.0.49 () {56 vars in 1148 bytes} [Wed Jun 15 10:22:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:22:03.192689 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf642b84-200a-4fae-b1b4-5ff5af47a489 None admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:22:03.289364 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf642b84-200a-4fae-b1b4-5ff5af47a489 None admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:22:03.322571 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d9382e-2361-4e2f-a038-479ba77b78a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:22:03.323248 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d9382e-2361-4e2f-a038-479ba77b78a8 None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:22:03.323536 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d9382e-2361-4e2f-a038-479ba77b78a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:22:03.323814 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d9382e-2361-4e2f-a038-479ba77b78a8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:22:03.324624 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d9382e-2361-4e2f-a038-479ba77b78a8 None admin] RBAC: Adding query filter params (role.id=8051174e7a7947d5b991ebd793c2e543, scope.project.id=559b3e713f5844b6ae37151d56dd419f, user.id=dd03612a525644518517069b3fa43522) {{(pid=80119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:22:03.324937 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d9382e-2361-4e2f-a038-479ba77b78a8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:22:03.326354 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d9382e-2361-4e2f-a038-479ba77b78a8 None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:22:03.332602 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 645/1287] 10.222.0.49 () {56 vars in 1374 bytes} [Wed Jun 15 10:22:03 2022] GET /identity/v3/role_assignments?user.id=dd03612a525644518517069b3fa43522&scope.project.id=559b3e713f5844b6ae37151d56dd419f&role.id=8051174e7a7947d5b991ebd793c2e543 => generated 608 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:22:05.449858 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77db5556-448b-4506-9d54-e6270ddc9fb1 None None] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:22:05.450422 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77db5556-448b-4506-9d54-e6270ddc9fb1 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:22:05.450422 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77db5556-448b-4506-9d54-e6270ddc9fb1 None None] PATH_INFO: `/` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:22:05.451801 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 643/1288] 10.222.0.49 () {54 vars in 926 bytes} [Wed Jun 15 10:22:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:22:05.457693 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaeb684b-3d5b-496b-ae32-5a2095193011 None None] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:22:05.458066 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaeb684b-3d5b-496b-ae32-5a2095193011 None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:22:05.458251 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaeb684b-3d5b-496b-ae32-5a2095193011 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:22:05.474671 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.auth.core [None req-eaeb684b-3d5b-496b-ae32-5a2095193011 None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:22:05.475902 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-eaeb684b-3d5b-496b-ae32-5a2095193011 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:22:05.504491 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=100 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:22:05.542775 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 646/1289] 10.222.0.49 () {58 vars in 1023 bytes} [Wed Jun 15 10:22:05 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 10:22:05.565896 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77db5556-448b-4506-9d54-e6270ddc9fb1 None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:22:05.672155 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77db5556-448b-4506-9d54-e6270ddc9fb1 None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:22:05.682252 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:22:05.704719 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25baa926-1909-4e22-8234-b98dd9eb99a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:22:05.705461 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25baa926-1909-4e22-8234-b98dd9eb99a1 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:22:05.705744 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25baa926-1909-4e22-8234-b98dd9eb99a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:22:05.706655 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25baa926-1909-4e22-8234-b98dd9eb99a1 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:22:05.707338 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25baa926-1909-4e22-8234-b98dd9eb99a1 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:22:05.708692 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25baa926-1909-4e22-8234-b98dd9eb99a1 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:22:05.720166 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application [None req-25baa926-1909-4e22-8234-b98dd9eb99a1 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jun 15 10:22:05.720166 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:22:05.720166 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 15 10:22:05.720166 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 15 10:22:05.720166 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 15 10:22:05.720166 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 15 10:22:05.720166 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 15 10:22:05.720166 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 10:22:05.720166 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 15 10:22:05.720166 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Jun 15 10:22:05.720166 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 15 10:22:05.720166 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Jun 15 10:22:05.720166 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:22:05.720166 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:22:05.720166 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:22:05.720166 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:22:05.720166 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:22:05.721398 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:22:05.721398 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:22:05.721398 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:22:05.721398 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:22:05.721398 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:22:05.721398 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:22:05.721398 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 15 10:22:05.721398 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 15 10:22:05.721398 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 15 10:22:05.721398 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 15 10:22:05.721398 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:22:05.721398 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:22:05.721398 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:22:05.721398 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:22:05.721398 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:22:05.721398 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:22:05.722515 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:22:05.722515 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application with Lock( Jun 15 10:22:05.722515 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:22:05.722515 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:22:05.722515 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:22:05.722515 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:22:05.722515 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:22:05.722515 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:22:05.722515 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:22:05.722515 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:22:05.722515 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 15 10:22:05.722515 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 10:22:05.722515 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 15 10:22:05.722515 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Jun 15 10:22:05.722515 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 644/1290] 10.222.0.49 () {56 vars in 1117 bytes} [Wed Jun 15 10:22:05 2022] GET /identity/v3/services/placement => generated 90 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:22:05.727402 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaeb684b-3d5b-496b-ae32-5a2095193011 None None] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:22:05.824618 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaeb684b-3d5b-496b-ae32-5a2095193011 None None] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:22:05.856527 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9d7da9-84ed-4af0-9ca0-7b555dfe7955 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:22:05.857182 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9d7da9-84ed-4af0-9ca0-7b555dfe7955 None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:22:05.857465 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9d7da9-84ed-4af0-9ca0-7b555dfe7955 None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:22:05.857739 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9d7da9-84ed-4af0-9ca0-7b555dfe7955 None admin] PATH_INFO: `/v3/services` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:22:05.858436 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9d7da9-84ed-4af0-9ca0-7b555dfe7955 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:22:05.858740 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9d7da9-84ed-4af0-9ca0-7b555dfe7955 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:22:05.860132 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9d7da9-84ed-4af0-9ca0-7b555dfe7955 None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:22:05.865324 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 647/1291] 10.222.0.49 () {56 vars in 1116 bytes} [Wed Jun 15 10:22:05 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 10:22:05.870930 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25baa926-1909-4e22-8234-b98dd9eb99a1 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:22:05.966419 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25baa926-1909-4e22-8234-b98dd9eb99a1 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:22:05.998782 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c857687c-8aae-4d78-9595-7ce880927b88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:22:05.999476 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c857687c-8aae-4d78-9595-7ce880927b88 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:22:05.999764 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c857687c-8aae-4d78-9595-7ce880927b88 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:22:06.000042 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c857687c-8aae-4d78-9595-7ce880927b88 None admin] PATH_INFO: `/v3/services` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:22:06.000755 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c857687c-8aae-4d78-9595-7ce880927b88 None admin] RBAC: Adding query filter params (type=placement) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:22:06.001063 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c857687c-8aae-4d78-9595-7ce880927b88 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:22:06.002436 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c857687c-8aae-4d78-9595-7ce880927b88 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:22:06.007575 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 645/1292] 10.222.0.49 () {56 vars in 1116 bytes} [Wed Jun 15 10:22:05 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:22:08.068814 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8a35e0-f670-4e5e-bccc-e3fd91b7c2c2 None None] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:22:08.069335 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8a35e0-f670-4e5e-bccc-e3fd91b7c2c2 None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:22:08.069335 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8a35e0-f670-4e5e-bccc-e3fd91b7c2c2 None None] PATH_INFO: `/` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:22:08.070634 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 648/1293] 10.222.0.49 () {54 vars in 926 bytes} [Wed Jun 15 10:22:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:22:08.075649 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63cd2fa5-f746-49d4-bb43-add9fd36fad8 None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:22:08.076053 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63cd2fa5-f746-49d4-bb43-add9fd36fad8 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:22:08.076207 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63cd2fa5-f746-49d4-bb43-add9fd36fad8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:22:08.093333 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-63cd2fa5-f746-49d4-bb43-add9fd36fad8 None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:22:08.094519 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-63cd2fa5-f746-49d4-bb43-add9fd36fad8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:22:08.123226 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:22:08.159719 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 646/1294] 10.222.0.49 () {58 vars in 1023 bytes} [Wed Jun 15 10:22:08 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 10:22:08.183027 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8a35e0-f670-4e5e-bccc-e3fd91b7c2c2 None None] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:22:08.213636 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=100 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:22:08.281568 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8a35e0-f670-4e5e-bccc-e3fd91b7c2c2 None None] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:22:08.314211 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30b122d-69b9-46bb-ac2e-7129a816348f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:22:08.314861 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30b122d-69b9-46bb-ac2e-7129a816348f None admin] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:22:08.315172 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30b122d-69b9-46bb-ac2e-7129a816348f None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:22:08.315986 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30b122d-69b9-46bb-ac2e-7129a816348f None admin] PATH_INFO: `/v3/services` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:22:08.316696 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30b122d-69b9-46bb-ac2e-7129a816348f None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:22:08.318054 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30b122d-69b9-46bb-ac2e-7129a816348f None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:22:08.330547 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 649/1295] 10.222.0.49 () {60 vars in 1141 bytes} [Wed Jun 15 10:22:08 2022] POST /identity/v3/services => generated 246 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 10:22:10.398125 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9931aa6c-917a-4b8e-b0a0-78e107415843 None None] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:22:10.398653 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9931aa6c-917a-4b8e-b0a0-78e107415843 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:22:10.398653 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9931aa6c-917a-4b8e-b0a0-78e107415843 None None] PATH_INFO: `/` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:22:10.400095 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 647/1296] 10.222.0.49 () {54 vars in 926 bytes} [Wed Jun 15 10:22:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:22:10.405222 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415db2d0-e331-4546-b261-fb99d9b0fb53 None None] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:22:10.405639 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415db2d0-e331-4546-b261-fb99d9b0fb53 None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:22:10.405877 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415db2d0-e331-4546-b261-fb99d9b0fb53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:22:10.422908 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.auth.core [None req-415db2d0-e331-4546-b261-fb99d9b0fb53 None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:22:10.424119 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-415db2d0-e331-4546-b261-fb99d9b0fb53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:22:10.492266 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 650/1297] 10.222.0.49 () {58 vars in 1023 bytes} [Wed Jun 15 10:22:10 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 10:22:10.514290 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9931aa6c-917a-4b8e-b0a0-78e107415843 None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:22:10.619113 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9931aa6c-917a-4b8e-b0a0-78e107415843 None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:22:10.652228 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f0b7a5-98fd-4a69-8135-b23fa88e664e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:22:10.652960 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f0b7a5-98fd-4a69-8135-b23fa88e664e None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:22:10.653242 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f0b7a5-98fd-4a69-8135-b23fa88e664e None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:22:10.653515 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f0b7a5-98fd-4a69-8135-b23fa88e664e None admin] PATH_INFO: `/v3/services/placement` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:22:10.654167 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f0b7a5-98fd-4a69-8135-b23fa88e664e None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:22:10.655561 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f0b7a5-98fd-4a69-8135-b23fa88e664e None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:22:10.667079 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application [None req-10f0b7a5-98fd-4a69-8135-b23fa88e664e None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jun 15 10:22:10.667079 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:22:10.667079 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 15 10:22:10.667079 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 15 10:22:10.667079 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 15 10:22:10.667079 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 15 10:22:10.667079 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 15 10:22:10.667079 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 10:22:10.667079 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 15 10:22:10.667079 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Jun 15 10:22:10.667079 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 15 10:22:10.667079 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Jun 15 10:22:10.667079 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:22:10.667079 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:22:10.667079 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:22:10.667079 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:22:10.667079 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:22:10.668262 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:22:10.668262 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:22:10.668262 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:22:10.668262 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:22:10.668262 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:22:10.668262 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:22:10.668262 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 15 10:22:10.668262 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 15 10:22:10.668262 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 15 10:22:10.668262 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 15 10:22:10.668262 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:22:10.668262 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:22:10.668262 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:22:10.668262 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:22:10.668262 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:22:10.668262 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:22:10.669149 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:22:10.669149 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application with Lock( Jun 15 10:22:10.669149 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:22:10.669149 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:22:10.669149 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:22:10.669149 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:22:10.669149 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:22:10.669149 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:22:10.669149 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:22:10.669149 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:22:10.669149 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 15 10:22:10.669149 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 10:22:10.669149 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 15 10:22:10.669149 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Jun 15 10:22:10.669149 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 648/1298] 10.222.0.49 () {56 vars in 1117 bytes} [Wed Jun 15 10:22:10 2022] GET /identity/v3/services/placement => generated 90 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:22:10.674502 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415db2d0-e331-4546-b261-fb99d9b0fb53 None None] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:22:10.692639 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:22:10.771217 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415db2d0-e331-4546-b261-fb99d9b0fb53 None None] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:22:10.805094 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faac3b2a-1b0a-4ce8-bd16-79c8b521acd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:22:10.805765 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faac3b2a-1b0a-4ce8-bd16-79c8b521acd7 None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:22:10.806053 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faac3b2a-1b0a-4ce8-bd16-79c8b521acd7 None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:22:10.806333 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faac3b2a-1b0a-4ce8-bd16-79c8b521acd7 None admin] PATH_INFO: `/v3/services` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:22:10.807039 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faac3b2a-1b0a-4ce8-bd16-79c8b521acd7 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:22:10.807380 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faac3b2a-1b0a-4ce8-bd16-79c8b521acd7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:22:10.808755 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faac3b2a-1b0a-4ce8-bd16-79c8b521acd7 None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:22:10.813797 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 651/1299] 10.222.0.49 () {56 vars in 1116 bytes} [Wed Jun 15 10:22:10 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:22:10.819874 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f0b7a5-98fd-4a69-8135-b23fa88e664e None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:22:10.850589 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:22:10.917803 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f0b7a5-98fd-4a69-8135-b23fa88e664e None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:22:10.950168 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2a7e74-6eb1-414a-a019-bcf9c31b3c8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:22:10.950812 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2a7e74-6eb1-414a-a019-bcf9c31b3c8b None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:22:10.951116 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2a7e74-6eb1-414a-a019-bcf9c31b3c8b None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:22:10.951401 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2a7e74-6eb1-414a-a019-bcf9c31b3c8b None admin] PATH_INFO: `/v3/endpoints` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:22:10.952185 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2a7e74-6eb1-414a-a019-bcf9c31b3c8b None admin] RBAC: Adding query filter params (interface=public, service_id=2b169ea1ec5f466fa03fdd5d9411376d, region_id=RegionOne) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:22:10.952494 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2a7e74-6eb1-414a-a019-bcf9c31b3c8b None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:22:10.953848 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2a7e74-6eb1-414a-a019-bcf9c31b3c8b None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:22:10.958733 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 649/1300] 10.222.0.49 () {56 vars in 1251 bytes} [Wed Jun 15 10:22:10 2022] GET /identity/v3/endpoints?service_id=2b169ea1ec5f466fa03fdd5d9411376d&interface=public®ion_id=RegionOne => generated 194 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:22:10.964335 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faac3b2a-1b0a-4ce8-bd16-79c8b521acd7 None admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:22:11.043326 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=100 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:22:11.061247 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faac3b2a-1b0a-4ce8-bd16-79c8b521acd7 None admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:22:11.093804 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d39792-a115-464d-9d52-818552a128bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:22:11.094462 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d39792-a115-464d-9d52-818552a128bb None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:22:11.094754 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d39792-a115-464d-9d52-818552a128bb None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:22:11.095035 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d39792-a115-464d-9d52-818552a128bb None admin] PATH_INFO: `/v3/services` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:22:11.095739 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d39792-a115-464d-9d52-818552a128bb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:22:11.097643 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d39792-a115-464d-9d52-818552a128bb None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:22:11.104224 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 652/1301] 10.222.0.49 () {56 vars in 1087 bytes} [Wed Jun 15 10:22:10 2022] GET /identity/v3/services => generated 2191 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 10:22:13.197852 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c3b5ca-aa48-4770-920a-0f4795ddc12b None None] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:22:13.198361 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c3b5ca-aa48-4770-920a-0f4795ddc12b None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:22:13.198361 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c3b5ca-aa48-4770-920a-0f4795ddc12b None None] PATH_INFO: `/` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:22:13.199741 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 650/1302] 10.222.0.49 () {54 vars in 926 bytes} [Wed Jun 15 10:22:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:22:13.204957 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ee88d3-2cba-4d9e-8f25-ffb594e5c577 None None] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:22:13.205349 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ee88d3-2cba-4d9e-8f25-ffb594e5c577 None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:22:13.205519 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ee88d3-2cba-4d9e-8f25-ffb594e5c577 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:22:13.222545 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.auth.core [None req-c2ee88d3-2cba-4d9e-8f25-ffb594e5c577 None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:22:13.223779 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-c2ee88d3-2cba-4d9e-8f25-ffb594e5c577 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:22:13.289195 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 653/1303] 10.222.0.49 () {58 vars in 1023 bytes} [Wed Jun 15 10:22:13 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 10:22:13.311498 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c3b5ca-aa48-4770-920a-0f4795ddc12b None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:22:13.403453 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:22:13.410758 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c3b5ca-aa48-4770-920a-0f4795ddc12b None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:22:13.442716 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1397569-a954-4a95-b18b-a9bd381862d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:22:13.443534 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1397569-a954-4a95-b18b-a9bd381862d6 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:22:13.443821 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1397569-a954-4a95-b18b-a9bd381862d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:22:13.444099 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1397569-a954-4a95-b18b-a9bd381862d6 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:22:13.444749 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1397569-a954-4a95-b18b-a9bd381862d6 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:22:13.446115 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1397569-a954-4a95-b18b-a9bd381862d6 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:22:13.458375 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application [None req-f1397569-a954-4a95-b18b-a9bd381862d6 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jun 15 10:22:13.458375 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:22:13.458375 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 15 10:22:13.458375 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 15 10:22:13.458375 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 15 10:22:13.458375 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 15 10:22:13.458375 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 15 10:22:13.458375 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 10:22:13.458375 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 15 10:22:13.458375 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Jun 15 10:22:13.458375 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 15 10:22:13.458375 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Jun 15 10:22:13.458375 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:22:13.458375 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:22:13.458375 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:22:13.458375 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:22:13.458375 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:22:13.459592 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:22:13.459592 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:22:13.459592 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:22:13.459592 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:22:13.459592 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:22:13.459592 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:22:13.459592 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 15 10:22:13.459592 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 15 10:22:13.459592 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 15 10:22:13.459592 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 15 10:22:13.459592 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:22:13.459592 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:22:13.459592 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:22:13.459592 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:22:13.459592 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:22:13.459592 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:22:13.460701 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:22:13.460701 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application with Lock( Jun 15 10:22:13.460701 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:22:13.460701 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:22:13.460701 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:22:13.460701 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:22:13.460701 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:22:13.460701 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:22:13.460701 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:22:13.460701 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:22:13.460701 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 15 10:22:13.460701 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 10:22:13.460701 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 15 10:22:13.460701 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Jun 15 10:22:13.460701 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 651/1304] 10.222.0.49 () {56 vars in 1117 bytes} [Wed Jun 15 10:22:13 2022] GET /identity/v3/services/placement => generated 90 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:22:13.465429 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ee88d3-2cba-4d9e-8f25-ffb594e5c577 None None] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:22:13.538648 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=100 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:22:13.560956 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ee88d3-2cba-4d9e-8f25-ffb594e5c577 None None] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:22:13.593219 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97bf6f9-f7ad-4c46-ad0e-bd0a0a18c08c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:22:13.593873 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97bf6f9-f7ad-4c46-ad0e-bd0a0a18c08c None admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:22:13.594157 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97bf6f9-f7ad-4c46-ad0e-bd0a0a18c08c None admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:22:13.594434 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97bf6f9-f7ad-4c46-ad0e-bd0a0a18c08c None admin] PATH_INFO: `/v3/services` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:22:13.595153 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d97bf6f9-f7ad-4c46-ad0e-bd0a0a18c08c None admin] RBAC: Adding query filter params (name=placement) {{(pid=80119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:22:13.595465 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d97bf6f9-f7ad-4c46-ad0e-bd0a0a18c08c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:22:13.597414 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d97bf6f9-f7ad-4c46-ad0e-bd0a0a18c08c None admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:22:13.602341 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 654/1305] 10.222.0.49 () {56 vars in 1116 bytes} [Wed Jun 15 10:22:13 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:22:13.607966 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1397569-a954-4a95-b18b-a9bd381862d6 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:22:13.703530 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1397569-a954-4a95-b18b-a9bd381862d6 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:22:13.735264 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc606d0-decc-4562-8e20-e5b6f7c9e3f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:22:13.735841 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc606d0-decc-4562-8e20-e5b6f7c9e3f6 None admin] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:22:13.736122 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc606d0-decc-4562-8e20-e5b6f7c9e3f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:22:13.736395 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc606d0-decc-4562-8e20-e5b6f7c9e3f6 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:22:13.737107 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc606d0-decc-4562-8e20-e5b6f7c9e3f6 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:22:13.738451 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc606d0-decc-4562-8e20-e5b6f7c9e3f6 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:22:13.755853 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 652/1306] 10.222.0.49 () {60 vars in 1144 bytes} [Wed Jun 15 10:22:13 2022] POST /identity/v3/endpoints => generated 331 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 10:22:23.599859 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=30 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:22:23.751150 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:23:15.895955 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2076d1e9-3011-41e9-a028-252778e032ef None None] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:23:15.896526 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2076d1e9-3011-41e9-a028-252778e032ef None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:23:15.896526 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2076d1e9-3011-41e9-a028-252778e032ef None None] PATH_INFO: `/` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:23:15.898277 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 655/1307] 10.222.0.49 () {54 vars in 926 bytes} [Wed Jun 15 10:23:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:23:15.904291 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177ed9e7-327f-4419-8872-996dadf6f0e8 None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:23:15.905075 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177ed9e7-327f-4419-8872-996dadf6f0e8 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:23:15.905619 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177ed9e7-327f-4419-8872-996dadf6f0e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:23:15.996411 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-177ed9e7-327f-4419-8872-996dadf6f0e8 None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:23:15.997922 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-177ed9e7-327f-4419-8872-996dadf6f0e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:23:16.229369 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 653/1308] 10.222.0.49 () {58 vars in 1023 bytes} [Wed Jun 15 10:23:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 314 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:23:16.253881 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1de1e71-55cd-4ae1-a7ae-f1d412cb96f7 None None] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:23:16.254261 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1de1e71-55cd-4ae1-a7ae-f1d412cb96f7 None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:23:16.254444 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1de1e71-55cd-4ae1-a7ae-f1d412cb96f7 None None] PATH_INFO: `/` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:23:16.256159 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 656/1309] 10.222.0.49 () {54 vars in 964 bytes} [Wed Jun 15 10:23:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:23:16.266968 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7705edf0-1a12-4130-b57d-9381b0b0c386 None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:23:16.267195 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7705edf0-1a12-4130-b57d-9381b0b0c386 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:23:16.267485 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7705edf0-1a12-4130-b57d-9381b0b0c386 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:23:16.364211 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-7705edf0-1a12-4130-b57d-9381b0b0c386 None None] MFA Rules not processed for user `0cb44a17d205431793e9a8791e477a32`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:23:16.366185 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-7705edf0-1a12-4130-b57d-9381b0b0c386 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:23:16.389243 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:23:16.451785 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 654/1310] 10.222.0.49 () {58 vars in 1061 bytes} [Wed Jun 15 10:23:16 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:23:16.461163 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1de1e71-55cd-4ae1-a7ae-f1d412cb96f7 None None] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:23:16.463237 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-e1de1e71-55cd-4ae1-a7ae-f1d412cb96f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:23:16.577353 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1de1e71-55cd-4ae1-a7ae-f1d412cb96f7 None None] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:23:16.652728 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dcdbe39-cf4c-44c6-9a2c-41333a6f24e5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0cb44a17d205431793e9a8791e477a32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '559b3e713f5844b6ae37151d56dd419f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:23:16.653397 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcdbe39-cf4c-44c6-9a2c-41333a6f24e5 service neutron] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:23:16.653759 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcdbe39-cf4c-44c6-9a2c-41333a6f24e5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:23:16.654111 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcdbe39-cf4c-44c6-9a2c-41333a6f24e5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:23:16.655889 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-1dcdbe39-cf4c-44c6-9a2c-41333a6f24e5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:23:16.709867 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=100 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:23:16.775059 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dcdbe39-cf4c-44c6-9a2c-41333a6f24e5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:23:16.776571 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dcdbe39-cf4c-44c6-9a2c-41333a6f24e5 service neutron] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:23:16.799555 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 657/1311] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Jun 15 10:23:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:23:19.217465 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed18ef8e-de79-45dc-92f6-6e21ee18de49 None None] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:23:19.218062 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed18ef8e-de79-45dc-92f6-6e21ee18de49 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:23:19.218062 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed18ef8e-de79-45dc-92f6-6e21ee18de49 None None] PATH_INFO: `/` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:23:19.219653 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 655/1312] 10.222.0.49 () {54 vars in 926 bytes} [Wed Jun 15 10:23:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:23:19.225188 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12524520-a4d5-476d-b479-3f5a1192cbfd None None] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:23:19.225711 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12524520-a4d5-476d-b479-3f5a1192cbfd None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:23:19.225793 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12524520-a4d5-476d-b479-3f5a1192cbfd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:23:19.306480 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.auth.core [None req-12524520-a4d5-476d-b479-3f5a1192cbfd None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:23:19.307814 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-12524520-a4d5-476d-b479-3f5a1192cbfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:23:19.418031 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 658/1313] 10.222.0.49 () {58 vars in 1023 bytes} [Wed Jun 15 10:23:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 10:23:19.457361 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed18ef8e-de79-45dc-92f6-6e21ee18de49 None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:23:19.458947 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-ed18ef8e-de79-45dc-92f6-6e21ee18de49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:23:19.581208 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed18ef8e-de79-45dc-92f6-6e21ee18de49 None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:23:19.657409 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:23:19.659865 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6042868-a827-4050-933d-0bf0841ea9f6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0cb44a17d205431793e9a8791e477a32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '559b3e713f5844b6ae37151d56dd419f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:23:19.660635 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6042868-a827-4050-933d-0bf0841ea9f6 service neutron] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:23:19.661128 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6042868-a827-4050-933d-0bf0841ea9f6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:23:19.661396 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6042868-a827-4050-933d-0bf0841ea9f6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:23:19.663734 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-e6042868-a827-4050-933d-0bf0841ea9f6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:23:19.772863 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6042868-a827-4050-933d-0bf0841ea9f6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:23:19.774241 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6042868-a827-4050-933d-0bf0841ea9f6 service neutron] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:23:19.794063 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 656/1314] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Jun 15 10:23:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:23:22.223123 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3074b9-be4c-4bfa-9f19-411e0a4286e7 None None] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:23:22.223659 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3074b9-be4c-4bfa-9f19-411e0a4286e7 None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:23:22.223755 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3074b9-be4c-4bfa-9f19-411e0a4286e7 None None] PATH_INFO: `/` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:23:22.225048 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 659/1315] 10.222.0.49 () {54 vars in 926 bytes} [Wed Jun 15 10:23:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:23:22.230275 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f7f2d7-b6f1-4392-a4a9-19ca922693a8 None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:23:22.230649 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f7f2d7-b6f1-4392-a4a9-19ca922693a8 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:23:22.230839 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f7f2d7-b6f1-4392-a4a9-19ca922693a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:23:22.294727 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-c6f7f2d7-b6f1-4392-a4a9-19ca922693a8 None None] MFA Rules not processed for user `2f093b1bbc6148f39242d0692e1e8a95`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:23:22.296060 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-c6f7f2d7-b6f1-4392-a4a9-19ca922693a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:23:22.347986 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:23:22.392559 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 657/1316] 10.222.0.49 () {58 vars in 1023 bytes} [Wed Jun 15 10:23:22 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:23:22.445476 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491a5378-dca3-4545-ab56-b32d775b0f72 None None] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:23:22.445770 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491a5378-dca3-4545-ab56-b32d775b0f72 None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:23:22.446053 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491a5378-dca3-4545-ab56-b32d775b0f72 None None] PATH_INFO: `/` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:23:22.447467 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 660/1317] 10.222.0.49 () {54 vars in 964 bytes} [Wed Jun 15 10:23:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:23:22.452993 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6b5e8d-4866-454d-8b48-468dff47d140 None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:23:22.453285 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6b5e8d-4866-454d-8b48-468dff47d140 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:23:22.453604 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6b5e8d-4866-454d-8b48-468dff47d140 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:23:22.525979 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-fc6b5e8d-4866-454d-8b48-468dff47d140 None None] MFA Rules not processed for user `0cb44a17d205431793e9a8791e477a32`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:23:22.527178 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-fc6b5e8d-4866-454d-8b48-468dff47d140 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:23:22.601912 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 658/1318] 10.222.0.49 () {58 vars in 1061 bytes} [Wed Jun 15 10:23:22 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:23:22.611459 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491a5378-dca3-4545-ab56-b32d775b0f72 None None] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:23:22.613005 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-491a5378-dca3-4545-ab56-b32d775b0f72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:23:22.622625 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=100 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:23:22.715023 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491a5378-dca3-4545-ab56-b32d775b0f72 None None] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:23:22.786870 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac6d76b-cec2-4581-858c-f4f0b200c450 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0cb44a17d205431793e9a8791e477a32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '559b3e713f5844b6ae37151d56dd419f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:23:22.787548 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac6d76b-cec2-4581-858c-f4f0b200c450 service neutron] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:23:22.787832 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac6d76b-cec2-4581-858c-f4f0b200c450 service neutron] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:23:22.788103 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac6d76b-cec2-4581-858c-f4f0b200c450 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:23:22.789747 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-7ac6d76b-cec2-4581-858c-f4f0b200c450 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:23:22.850441 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=100 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:23:22.895146 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac6d76b-cec2-4581-858c-f4f0b200c450 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:23:22.896586 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac6d76b-cec2-4581-858c-f4f0b200c450 service neutron] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:23:22.914853 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 661/1319] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Jun 15 10:23:22 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:23:25.926266 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062d7a09-4a3d-42be-953c-b40c0ec58556 None None] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:23:25.926772 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062d7a09-4a3d-42be-953c-b40c0ec58556 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:23:25.926772 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062d7a09-4a3d-42be-953c-b40c0ec58556 None None] PATH_INFO: `/` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:23:25.928167 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 659/1320] 10.222.0.49 () {54 vars in 926 bytes} [Wed Jun 15 10:23:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:23:25.933289 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9ccf1e-7130-444a-a5a9-20f89a3d7be2 None None] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:23:25.933759 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9ccf1e-7130-444a-a5a9-20f89a3d7be2 None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:23:25.933852 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9ccf1e-7130-444a-a5a9-20f89a3d7be2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:23:25.995987 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.auth.core [None req-3e9ccf1e-7130-444a-a5a9-20f89a3d7be2 None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:23:25.997187 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-3e9ccf1e-7130-444a-a5a9-20f89a3d7be2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:23:26.091099 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 662/1321] 10.222.0.49 () {58 vars in 1023 bytes} [Wed Jun 15 10:23:25 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:23:26.129422 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062d7a09-4a3d-42be-953c-b40c0ec58556 None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:23:26.130980 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-062d7a09-4a3d-42be-953c-b40c0ec58556 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:23:26.181622 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:23:26.234437 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062d7a09-4a3d-42be-953c-b40c0ec58556 None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:23:26.305424 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d273bb8-da15-4ed3-b6a1-34da9f50953d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0cb44a17d205431793e9a8791e477a32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '559b3e713f5844b6ae37151d56dd419f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:23:26.306125 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d273bb8-da15-4ed3-b6a1-34da9f50953d service neutron] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:23:26.306660 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d273bb8-da15-4ed3-b6a1-34da9f50953d service neutron] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:23:26.306945 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d273bb8-da15-4ed3-b6a1-34da9f50953d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:23:26.308614 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-4d273bb8-da15-4ed3-b6a1-34da9f50953d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:23:26.402871 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:23:26.416155 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d273bb8-da15-4ed3-b6a1-34da9f50953d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:23:26.417591 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d273bb8-da15-4ed3-b6a1-34da9f50953d service neutron] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:23:26.435552 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 660/1322] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Jun 15 10:23:26 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:23:29.231958 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b4962c-ee55-4e04-8203-3bc583affada None None] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:23:29.232529 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b4962c-ee55-4e04-8203-3bc583affada None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:23:29.232529 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b4962c-ee55-4e04-8203-3bc583affada None None] PATH_INFO: `/` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:23:29.233843 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 663/1323] 10.222.0.49 () {54 vars in 926 bytes} [Wed Jun 15 10:23:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:23:29.239688 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f99ebf4-87cb-40e1-8961-1df0e5fc205e None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:23:29.240245 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f99ebf4-87cb-40e1-8961-1df0e5fc205e None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:23:29.240245 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f99ebf4-87cb-40e1-8961-1df0e5fc205e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:23:29.311066 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-8f99ebf4-87cb-40e1-8961-1df0e5fc205e None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:23:29.312342 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-8f99ebf4-87cb-40e1-8961-1df0e5fc205e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:23:29.413303 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 661/1324] 10.222.0.49 () {58 vars in 1023 bytes} [Wed Jun 15 10:23:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:23:29.448926 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b4962c-ee55-4e04-8203-3bc583affada None None] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:23:29.450516 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-22b4962c-ee55-4e04-8203-3bc583affada None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:23:29.469866 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=100 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:23:29.557216 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b4962c-ee55-4e04-8203-3bc583affada None None] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:23:29.630213 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb3304e-580a-4433-afe0-029332019fd2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0cb44a17d205431793e9a8791e477a32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '559b3e713f5844b6ae37151d56dd419f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:23:29.630867 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb3304e-580a-4433-afe0-029332019fd2 service neutron] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:23:29.631187 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb3304e-580a-4433-afe0-029332019fd2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:23:29.631462 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb3304e-580a-4433-afe0-029332019fd2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:23:29.633163 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-efb3304e-580a-4433-afe0-029332019fd2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:23:29.702368 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=100 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:23:29.741125 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb3304e-580a-4433-afe0-029332019fd2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:23:29.742578 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb3304e-580a-4433-afe0-029332019fd2 service neutron] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:23:29.760344 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 664/1325] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Jun 15 10:23:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:23:31.932996 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb3703d-915c-4a8b-baca-4ea8a069a988 None None] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:23:31.933560 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb3703d-915c-4a8b-baca-4ea8a069a988 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:23:31.933560 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb3703d-915c-4a8b-baca-4ea8a069a988 None None] PATH_INFO: `/` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:23:31.935109 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 662/1326] 10.222.0.49 () {54 vars in 926 bytes} [Wed Jun 15 10:23:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:23:31.940627 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295e6dd6-e768-46ce-a1fd-980d00ee9f30 None None] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:23:31.941028 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295e6dd6-e768-46ce-a1fd-980d00ee9f30 None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:23:31.941186 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295e6dd6-e768-46ce-a1fd-980d00ee9f30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:23:32.005025 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.auth.core [None req-295e6dd6-e768-46ce-a1fd-980d00ee9f30 None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:23:32.006240 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-295e6dd6-e768-46ce-a1fd-980d00ee9f30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:23:32.100480 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 665/1327] 10.222.0.49 () {58 vars in 1023 bytes} [Wed Jun 15 10:23:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:23:32.136715 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb3703d-915c-4a8b-baca-4ea8a069a988 None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:23:32.138276 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-abb3703d-915c-4a8b-baca-4ea8a069a988 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:23:32.201110 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:23:32.245573 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb3703d-915c-4a8b-baca-4ea8a069a988 None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:23:32.319544 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f11477c-10ed-431f-b903-a8740efd553f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0cb44a17d205431793e9a8791e477a32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '559b3e713f5844b6ae37151d56dd419f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:23:32.320203 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f11477c-10ed-431f-b903-a8740efd553f service neutron] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:23:32.320489 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f11477c-10ed-431f-b903-a8740efd553f service neutron] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:23:32.320764 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f11477c-10ed-431f-b903-a8740efd553f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:23:32.322414 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-6f11477c-10ed-431f-b903-a8740efd553f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:23:32.436636 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:23:32.439075 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f11477c-10ed-431f-b903-a8740efd553f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:23:32.440880 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f11477c-10ed-431f-b903-a8740efd553f service neutron] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:23:32.458995 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 663/1328] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Jun 15 10:23:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:23:34.628475 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d42d85-4eb8-489d-b6cf-98cacdcace4c None None] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:23:34.629022 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d42d85-4eb8-489d-b6cf-98cacdcace4c None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:23:34.629185 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d42d85-4eb8-489d-b6cf-98cacdcace4c None None] PATH_INFO: `/` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:23:34.630643 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 666/1329] 10.222.0.49 () {54 vars in 926 bytes} [Wed Jun 15 10:23:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:23:34.636022 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6af938-a020-415a-b0af-a6cc18767b6e None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:23:34.636476 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6af938-a020-415a-b0af-a6cc18767b6e None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:23:34.636689 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6af938-a020-415a-b0af-a6cc18767b6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:23:34.700143 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-4d6af938-a020-415a-b0af-a6cc18767b6e None None] MFA Rules not processed for user `2f093b1bbc6148f39242d0692e1e8a95`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:23:34.701355 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-4d6af938-a020-415a-b0af-a6cc18767b6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:23:34.800088 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 664/1330] 10.222.0.49 () {58 vars in 1023 bytes} [Wed Jun 15 10:23:34 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:23:34.836197 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d42d85-4eb8-489d-b6cf-98cacdcace4c None None] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:23:34.837743 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-c5d42d85-4eb8-489d-b6cf-98cacdcace4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:23:34.871014 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=100 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:23:34.961745 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d42d85-4eb8-489d-b6cf-98cacdcace4c None None] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:23:35.038200 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83fe81f5-e90c-41c3-b701-9378318d9f8b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0cb44a17d205431793e9a8791e477a32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '559b3e713f5844b6ae37151d56dd419f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:23:35.038861 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fe81f5-e90c-41c3-b701-9378318d9f8b service neutron] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:23:35.039178 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fe81f5-e90c-41c3-b701-9378318d9f8b service neutron] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:23:35.039459 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fe81f5-e90c-41c3-b701-9378318d9f8b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:23:35.041105 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-83fe81f5-e90c-41c3-b701-9378318d9f8b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:23:35.117711 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=100 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:23:35.149307 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83fe81f5-e90c-41c3-b701-9378318d9f8b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:23:35.150721 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83fe81f5-e90c-41c3-b701-9378318d9f8b service neutron] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:23:35.169302 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 667/1331] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Jun 15 10:23:34 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:23:37.523912 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf7a53a-bf55-4836-b199-714c949edaf1 None None] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:23:37.524454 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf7a53a-bf55-4836-b199-714c949edaf1 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:23:37.524566 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf7a53a-bf55-4836-b199-714c949edaf1 None None] PATH_INFO: `/` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:23:37.525880 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 665/1332] 10.222.0.49 () {54 vars in 926 bytes} [Wed Jun 15 10:23:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:23:37.530977 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d478bc-ea78-4621-8c21-375fc5423ede None None] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:23:37.531419 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d478bc-ea78-4621-8c21-375fc5423ede None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:23:37.531579 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d478bc-ea78-4621-8c21-375fc5423ede None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:23:37.596863 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.auth.core [None req-60d478bc-ea78-4621-8c21-375fc5423ede None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:23:37.598100 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-60d478bc-ea78-4621-8c21-375fc5423ede None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:23:37.694847 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 668/1333] 10.222.0.49 () {58 vars in 1023 bytes} [Wed Jun 15 10:23:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:23:37.730547 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf7a53a-bf55-4836-b199-714c949edaf1 None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:23:37.732118 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-1bf7a53a-bf55-4836-b199-714c949edaf1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:23:37.804795 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:23:37.839176 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf7a53a-bf55-4836-b199-714c949edaf1 None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:23:37.910480 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28ef980-c033-4d33-bfa8-5aeb9149c71f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0cb44a17d205431793e9a8791e477a32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '559b3e713f5844b6ae37151d56dd419f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:23:37.911210 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28ef980-c033-4d33-bfa8-5aeb9149c71f service neutron] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:23:37.911499 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28ef980-c033-4d33-bfa8-5aeb9149c71f service neutron] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:23:37.911779 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28ef980-c033-4d33-bfa8-5aeb9149c71f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:23:37.913437 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-e28ef980-c033-4d33-bfa8-5aeb9149c71f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:23:38.020850 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e28ef980-c033-4d33-bfa8-5aeb9149c71f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:23:38.022276 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e28ef980-c033-4d33-bfa8-5aeb9149c71f service neutron] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:23:38.032076 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:23:38.040314 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 666/1334] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Jun 15 10:23:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:23:40.169145 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdc4eb7-d5d4-45da-b180-ecd8eebae74d None None] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:23:40.169686 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdc4eb7-d5d4-45da-b180-ecd8eebae74d None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:23:40.169686 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdc4eb7-d5d4-45da-b180-ecd8eebae74d None None] PATH_INFO: `/` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:23:40.171051 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 669/1335] 10.222.0.49 () {54 vars in 926 bytes} [Wed Jun 15 10:23:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:23:40.176091 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c55330f-dbf5-4c78-88be-971b4a28876e None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:23:40.176514 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c55330f-dbf5-4c78-88be-971b4a28876e None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:23:40.176649 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c55330f-dbf5-4c78-88be-971b4a28876e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:23:40.239867 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-3c55330f-dbf5-4c78-88be-971b4a28876e None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:23:40.241054 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-3c55330f-dbf5-4c78-88be-971b4a28876e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:23:40.334884 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 667/1336] 10.222.0.49 () {58 vars in 1023 bytes} [Wed Jun 15 10:23:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:23:40.373928 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecdc4eb7-d5d4-45da-b180-ecd8eebae74d None None] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:23:40.375506 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-ecdc4eb7-d5d4-45da-b180-ecd8eebae74d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:23:40.415321 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=100 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:23:40.479147 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecdc4eb7-d5d4-45da-b180-ecd8eebae74d None None] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:23:40.552428 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f1940a-8e54-41b4-acf5-54677bf59bb0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0cb44a17d205431793e9a8791e477a32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '559b3e713f5844b6ae37151d56dd419f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:23:40.553099 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f1940a-8e54-41b4-acf5-54677bf59bb0 service neutron] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:23:40.553385 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f1940a-8e54-41b4-acf5-54677bf59bb0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:23:40.553662 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f1940a-8e54-41b4-acf5-54677bf59bb0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:23:40.555365 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-d4f1940a-8e54-41b4-acf5-54677bf59bb0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:23:40.635953 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=100 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:23:40.661608 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f1940a-8e54-41b4-acf5-54677bf59bb0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:23:40.663046 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f1940a-8e54-41b4-acf5-54677bf59bb0 service neutron] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:23:40.681402 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 670/1337] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Jun 15 10:23:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:23:43.248094 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc27f9a8-9030-4619-a834-5e0fd77c2904 None None] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:23:43.248627 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc27f9a8-9030-4619-a834-5e0fd77c2904 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:23:43.248627 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc27f9a8-9030-4619-a834-5e0fd77c2904 None None] PATH_INFO: `/` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:23:43.249929 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 668/1338] 10.222.0.49 () {54 vars in 926 bytes} [Wed Jun 15 10:23:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:23:43.254991 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d068f2c4-321c-4075-a7b6-c9177a0c0f89 None None] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:23:43.255453 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d068f2c4-321c-4075-a7b6-c9177a0c0f89 None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:23:43.255564 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d068f2c4-321c-4075-a7b6-c9177a0c0f89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:23:43.321014 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.auth.core [None req-d068f2c4-321c-4075-a7b6-c9177a0c0f89 None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:23:43.322218 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-d068f2c4-321c-4075-a7b6-c9177a0c0f89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:23:43.415595 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 671/1339] 10.222.0.49 () {58 vars in 1023 bytes} [Wed Jun 15 10:23:43 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:23:43.450696 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc27f9a8-9030-4619-a834-5e0fd77c2904 None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:23:43.452253 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-dc27f9a8-9030-4619-a834-5e0fd77c2904 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:23:43.525857 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:23:43.554973 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc27f9a8-9030-4619-a834-5e0fd77c2904 None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:23:43.630542 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e518259b-b475-48c8-90ad-de908bcec073 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0cb44a17d205431793e9a8791e477a32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '559b3e713f5844b6ae37151d56dd419f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:23:43.631225 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e518259b-b475-48c8-90ad-de908bcec073 service neutron] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:23:43.631509 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e518259b-b475-48c8-90ad-de908bcec073 service neutron] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:23:43.631784 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e518259b-b475-48c8-90ad-de908bcec073 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:23:43.633433 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-e518259b-b475-48c8-90ad-de908bcec073 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:23:43.738855 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e518259b-b475-48c8-90ad-de908bcec073 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:23:43.740312 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e518259b-b475-48c8-90ad-de908bcec073 service neutron] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:23:43.758462 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 669/1340] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Jun 15 10:23:43 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:23:47.911209 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0704b8-7367-45f9-9abd-d8dba5ae0ad5 None None] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:23:47.911736 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0704b8-7367-45f9-9abd-d8dba5ae0ad5 None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:23:47.911736 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0704b8-7367-45f9-9abd-d8dba5ae0ad5 None None] PATH_INFO: `/` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:23:47.913194 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 672/1341] 10.222.0.49 () {54 vars in 926 bytes} [Wed Jun 15 10:23:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:23:47.918537 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcaf012-80e0-4fe2-b010-792e48a1e702 None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:23:47.918984 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcaf012-80e0-4fe2-b010-792e48a1e702 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:23:47.919114 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcaf012-80e0-4fe2-b010-792e48a1e702 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:23:47.926178 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:23:47.993145 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-8bcaf012-80e0-4fe2-b010-792e48a1e702 None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:23:47.994332 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-8bcaf012-80e0-4fe2-b010-792e48a1e702 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:23:48.099219 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 670/1342] 10.222.0.49 () {58 vars in 1023 bytes} [Wed Jun 15 10:23:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:23:48.138033 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0704b8-7367-45f9-9abd-d8dba5ae0ad5 None None] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:23:48.140494 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-4f0704b8-7367-45f9-9abd-d8dba5ae0ad5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:23:48.191212 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=100 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:23:48.238230 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0704b8-7367-45f9-9abd-d8dba5ae0ad5 None None] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:23:48.241780 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3830576b-851f-464b-a79a-8e14c508c5e8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0cb44a17d205431793e9a8791e477a32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '559b3e713f5844b6ae37151d56dd419f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:23:48.242430 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3830576b-851f-464b-a79a-8e14c508c5e8 service neutron] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:23:48.242712 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3830576b-851f-464b-a79a-8e14c508c5e8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:23:48.242983 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3830576b-851f-464b-a79a-8e14c508c5e8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:23:48.252698 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3830576b-851f-464b-a79a-8e14c508c5e8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:23:48.254135 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3830576b-851f-464b-a79a-8e14c508c5e8 service neutron] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:23:48.263924 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 673/1343] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Jun 15 10:23:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:23:50.996659 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e16b9c-a07f-4b64-9796-c7ddb950c581 None None] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:23:50.997214 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e16b9c-a07f-4b64-9796-c7ddb950c581 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:23:50.997214 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e16b9c-a07f-4b64-9796-c7ddb950c581 None None] PATH_INFO: `/` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:23:50.998636 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 671/1344] 10.222.0.49 () {54 vars in 926 bytes} [Wed Jun 15 10:23:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:23:51.004546 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209bab78-b9f6-4f41-a361-31f799bca3d5 None None] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:23:51.005121 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209bab78-b9f6-4f41-a361-31f799bca3d5 None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:23:51.005121 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209bab78-b9f6-4f41-a361-31f799bca3d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:23:51.036238 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.auth.core [None req-209bab78-b9f6-4f41-a361-31f799bca3d5 None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:23:51.037481 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-209bab78-b9f6-4f41-a361-31f799bca3d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:23:51.120426 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 674/1345] 10.222.0.49 () {58 vars in 1023 bytes} [Wed Jun 15 10:23:51 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:23:51.158136 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e16b9c-a07f-4b64-9796-c7ddb950c581 None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:23:51.169630 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e16b9c-a07f-4b64-9796-c7ddb950c581 None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:23:51.173426 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227dedfc-aaa6-423e-bde4-16603bcc1914 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0cb44a17d205431793e9a8791e477a32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '559b3e713f5844b6ae37151d56dd419f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:23:51.174166 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227dedfc-aaa6-423e-bde4-16603bcc1914 service neutron] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:23:51.174521 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227dedfc-aaa6-423e-bde4-16603bcc1914 service neutron] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:23:51.174862 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227dedfc-aaa6-423e-bde4-16603bcc1914 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:23:51.184648 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227dedfc-aaa6-423e-bde4-16603bcc1914 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:23:51.186082 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227dedfc-aaa6-423e-bde4-16603bcc1914 service neutron] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:23:51.196320 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 672/1346] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Jun 15 10:23:51 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:23:55.218471 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62e15de-3d36-4fac-9712-65ea60a3c8ce None None] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:23:55.218961 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62e15de-3d36-4fac-9712-65ea60a3c8ce None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:23:55.219079 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62e15de-3d36-4fac-9712-65ea60a3c8ce None None] PATH_INFO: `/` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:23:55.221035 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 675/1347] 10.222.0.49 () {54 vars in 926 bytes} [Wed Jun 15 10:23:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:23:55.226473 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3abaf8-47e8-43e5-847e-aa30c8e1ae25 None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:23:55.227313 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3abaf8-47e8-43e5-847e-aa30c8e1ae25 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:23:55.227597 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3abaf8-47e8-43e5-847e-aa30c8e1ae25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:23:55.245114 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-ae3abaf8-47e8-43e5-847e-aa30c8e1ae25 None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:23:55.246354 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-ae3abaf8-47e8-43e5-847e-aa30c8e1ae25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:23:55.254234 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 673/1348] 10.222.0.49 () {58 vars in 1023 bytes} [Wed Jun 15 10:23:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:23:55.291971 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62e15de-3d36-4fac-9712-65ea60a3c8ce None None] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:23:55.303171 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62e15de-3d36-4fac-9712-65ea60a3c8ce None None] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:23:55.306750 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71ebf35-bf24-4411-a2d5-0dfab4ebd4b2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0cb44a17d205431793e9a8791e477a32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '559b3e713f5844b6ae37151d56dd419f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:23:55.307367 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71ebf35-bf24-4411-a2d5-0dfab4ebd4b2 service neutron] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:23:55.307656 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71ebf35-bf24-4411-a2d5-0dfab4ebd4b2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:23:55.307939 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71ebf35-bf24-4411-a2d5-0dfab4ebd4b2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:23:55.316891 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71ebf35-bf24-4411-a2d5-0dfab4ebd4b2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:23:55.318324 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71ebf35-bf24-4411-a2d5-0dfab4ebd4b2 service neutron] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:23:55.327955 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 676/1349] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Jun 15 10:23:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 10:24:00.427526 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a03f03-af66-4cd0-b89e-3466c8d1881b None None] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:24:00.428423 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a03f03-af66-4cd0-b89e-3466c8d1881b None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:24:00.428875 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a03f03-af66-4cd0-b89e-3466c8d1881b None None] PATH_INFO: `/` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:24:00.430682 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 674/1350] 10.222.0.49 () {54 vars in 921 bytes} [Wed Jun 15 10:24:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:24:00.437138 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f687647-6053-4a74-a868-bc53d5874cd1 None None] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:24:00.437616 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f687647-6053-4a74-a868-bc53d5874cd1 None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:24:00.437956 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f687647-6053-4a74-a868-bc53d5874cd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:24:00.518146 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.auth.core [None req-9f687647-6053-4a74-a868-bc53d5874cd1 None None] MFA Rules not processed for user `dd03612a525644518517069b3fa43522`. Rule list: `[]` (Enabled: `True`). {{(pid=80119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:24:00.519366 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-9f687647-6053-4a74-a868-bc53d5874cd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:24:00.550964 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=100 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:24:00.606800 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 677/1351] 10.222.0.49 () {58 vars in 1018 bytes} [Wed Jun 15 10:24:00 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:24:04.772275 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2494930-234c-4e39-b630-49eb08d131c6 None None] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:24:04.772275 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2494930-234c-4e39-b630-49eb08d131c6 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:24:04.772275 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2494930-234c-4e39-b630-49eb08d131c6 None None] PATH_INFO: `/` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:24:04.772275 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 675/1352] 10.222.0.49 () {54 vars in 921 bytes} [Wed Jun 15 10:24:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:24:04.776797 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c375cf-d6fb-4862-92b3-29a764b8a5a2 None None] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:24:04.777222 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c375cf-d6fb-4862-92b3-29a764b8a5a2 None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:24:04.777415 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c375cf-d6fb-4862-92b3-29a764b8a5a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:24:04.796131 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.auth.core [None req-d7c375cf-d6fb-4862-92b3-29a764b8a5a2 None None] MFA Rules not processed for user `dd03612a525644518517069b3fa43522`. Rule list: `[]` (Enabled: `True`). {{(pid=80119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:24:04.797976 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-d7c375cf-d6fb-4862-92b3-29a764b8a5a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:24:04.806660 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 678/1353] 10.222.0.49 () {58 vars in 1018 bytes} [Wed Jun 15 10:24:04 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:24:05.240518 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=72 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:24:06.397944 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9282c68d-7bef-46c9-86b3-a178d5a87c99 None None] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:24:06.398618 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9282c68d-7bef-46c9-86b3-a178d5a87c99 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:24:06.398618 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9282c68d-7bef-46c9-86b3-a178d5a87c99 None None] PATH_INFO: `/` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:24:06.399958 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 676/1354] 10.222.0.49 () {54 vars in 921 bytes} [Wed Jun 15 10:24:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:24:06.405436 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a125f5-d459-41ea-ac9a-a051851e9edd None None] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:24:06.405868 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a125f5-d459-41ea-ac9a-a051851e9edd None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:24:06.406101 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a125f5-d459-41ea-ac9a-a051851e9edd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:24:06.425300 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.auth.core [None req-d5a125f5-d459-41ea-ac9a-a051851e9edd None None] MFA Rules not processed for user `dd03612a525644518517069b3fa43522`. Rule list: `[]` (Enabled: `True`). {{(pid=80119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:24:06.426479 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-d5a125f5-d459-41ea-ac9a-a051851e9edd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:24:06.434864 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 679/1355] 10.222.0.49 () {58 vars in 1018 bytes} [Wed Jun 15 10:24:06 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:24:10.109647 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1530ab5d-1b7a-460b-9672-e80d860948cc None None] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:24:10.110160 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1530ab5d-1b7a-460b-9672-e80d860948cc None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:24:10.110337 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1530ab5d-1b7a-460b-9672-e80d860948cc None None] PATH_INFO: `/` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:24:10.111858 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 677/1356] 10.222.0.49 () {54 vars in 926 bytes} [Wed Jun 15 10:24:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:24:10.117036 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbcf99e-c4ad-451e-9e86-5ef04b2e673c None None] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:24:10.117438 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbcf99e-c4ad-451e-9e86-5ef04b2e673c None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:24:10.117586 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbcf99e-c4ad-451e-9e86-5ef04b2e673c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:24:10.136006 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.auth.core [None req-7bbcf99e-c4ad-451e-9e86-5ef04b2e673c None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:24:10.137194 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-7bbcf99e-c4ad-451e-9e86-5ef04b2e673c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:24:10.145667 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 680/1357] 10.222.0.49 () {58 vars in 1023 bytes} [Wed Jun 15 10:24:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:24:10.200120 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2857c078-2f0f-40ba-b057-076ae014281f None None] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:24:10.200795 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2857c078-2f0f-40ba-b057-076ae014281f None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:24:10.201422 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2857c078-2f0f-40ba-b057-076ae014281f None None] PATH_INFO: `/` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:24:10.204293 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 678/1358] 10.222.0.49 () {54 vars in 961 bytes} [Wed Jun 15 10:24:10 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:24:10.210999 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6a1351-163f-4000-b141-94d0100f364d None None] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:24:10.211334 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6a1351-163f-4000-b141-94d0100f364d None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:24:10.211611 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6a1351-163f-4000-b141-94d0100f364d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:24:10.263895 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.auth.core [None req-bd6a1351-163f-4000-b141-94d0100f364d None None] MFA Rules not processed for user `5ac54528370d4747b6e8777fba9d0c82`. Rule list: `[]` (Enabled: `True`). {{(pid=80119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:24:10.265292 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-bd6a1351-163f-4000-b141-94d0100f364d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:24:10.349636 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 681/1359] 10.222.0.49 () {58 vars in 1058 bytes} [Wed Jun 15 10:24:10 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:24:10.357493 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2857c078-2f0f-40ba-b057-076ae014281f None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:24:10.372342 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2857c078-2f0f-40ba-b057-076ae014281f None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:24:10.378395 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97471e2f-7c8f-4028-897d-c2cf951bd514 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ac54528370d4747b6e8777fba9d0c82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '559b3e713f5844b6ae37151d56dd419f', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:24:10.379167 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97471e2f-7c8f-4028-897d-c2cf951bd514 service nova] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:24:10.379547 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97471e2f-7c8f-4028-897d-c2cf951bd514 service nova] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:24:10.379923 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97471e2f-7c8f-4028-897d-c2cf951bd514 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:24:10.388748 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97471e2f-7c8f-4028-897d-c2cf951bd514 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:24:10.390502 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97471e2f-7c8f-4028-897d-c2cf951bd514 service nova] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:24:10.401053 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 679/1360] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Jun 15 10:24:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:24:12.711807 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ec62db-f042-4af9-aa85-d23c722cf19c None None] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:24:12.712365 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ec62db-f042-4af9-aa85-d23c722cf19c None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:24:12.712365 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ec62db-f042-4af9-aa85-d23c722cf19c None None] PATH_INFO: `/` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:24:12.713904 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 682/1361] 10.222.0.49 () {54 vars in 926 bytes} [Wed Jun 15 10:24:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:24:12.718976 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819b04e7-f5d7-4caa-ae0e-b0d93d316115 None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:24:12.719437 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819b04e7-f5d7-4caa-ae0e-b0d93d316115 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:24:12.719739 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819b04e7-f5d7-4caa-ae0e-b0d93d316115 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:24:12.738316 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-819b04e7-f5d7-4caa-ae0e-b0d93d316115 None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:24:12.739577 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-819b04e7-f5d7-4caa-ae0e-b0d93d316115 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:24:12.747658 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 680/1362] 10.222.0.49 () {58 vars in 1023 bytes} [Wed Jun 15 10:24:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:24:12.793663 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ec62db-f042-4af9-aa85-d23c722cf19c None None] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:24:12.810161 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ec62db-f042-4af9-aa85-d23c722cf19c None None] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:24:12.816725 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc7c461-c024-467e-a596-92ea240af653 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ac54528370d4747b6e8777fba9d0c82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '559b3e713f5844b6ae37151d56dd419f', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:24:12.817537 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc7c461-c024-467e-a596-92ea240af653 service nova] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:24:12.817897 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc7c461-c024-467e-a596-92ea240af653 service nova] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:24:12.818251 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc7c461-c024-467e-a596-92ea240af653 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:24:12.828140 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc7c461-c024-467e-a596-92ea240af653 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:24:12.829883 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc7c461-c024-467e-a596-92ea240af653 service nova] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:24:12.841165 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 683/1363] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Jun 15 10:24:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:24:15.281048 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64a25a2-7900-4e43-bc85-275a8274a3f9 None None] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:24:15.281613 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64a25a2-7900-4e43-bc85-275a8274a3f9 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:24:15.281613 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64a25a2-7900-4e43-bc85-275a8274a3f9 None None] PATH_INFO: `/` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:24:15.282987 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 681/1364] 10.222.0.49 () {54 vars in 926 bytes} [Wed Jun 15 10:24:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:24:15.288212 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22e8253-5573-44c4-a170-08155f231ebc None None] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:24:15.288635 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22e8253-5573-44c4-a170-08155f231ebc None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:24:15.288781 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22e8253-5573-44c4-a170-08155f231ebc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:24:15.307230 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.auth.core [None req-a22e8253-5573-44c4-a170-08155f231ebc None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:24:15.308673 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-a22e8253-5573-44c4-a170-08155f231ebc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:24:15.317113 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 684/1365] 10.222.0.49 () {58 vars in 1023 bytes} [Wed Jun 15 10:24:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:24:15.361840 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dac404b-5a2d-4763-b018-dd935321966e None None] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:24:15.362130 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dac404b-5a2d-4763-b018-dd935321966e None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:24:15.362975 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dac404b-5a2d-4763-b018-dd935321966e None None] PATH_INFO: `/` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:24:15.364682 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 682/1366] 10.222.0.49 () {54 vars in 961 bytes} [Wed Jun 15 10:24:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:24:15.369716 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4c12f3-32ff-40b3-b4e3-7922ac4c6142 None None] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:24:15.370148 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4c12f3-32ff-40b3-b4e3-7922ac4c6142 None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:24:15.370462 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4c12f3-32ff-40b3-b4e3-7922ac4c6142 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:24:15.388837 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.auth.core [None req-cf4c12f3-32ff-40b3-b4e3-7922ac4c6142 None None] MFA Rules not processed for user `5ac54528370d4747b6e8777fba9d0c82`. Rule list: `[]` (Enabled: `True`). {{(pid=80119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:24:15.390058 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-cf4c12f3-32ff-40b3-b4e3-7922ac4c6142 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:24:15.406382 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 685/1367] 10.222.0.49 () {58 vars in 1058 bytes} [Wed Jun 15 10:24:15 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:24:15.414107 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dac404b-5a2d-4763-b018-dd935321966e None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:24:15.428876 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dac404b-5a2d-4763-b018-dd935321966e None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:24:15.434237 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff7a81b-f401-4eab-b096-a68bb961c856 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ac54528370d4747b6e8777fba9d0c82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '559b3e713f5844b6ae37151d56dd419f', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:24:15.434874 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff7a81b-f401-4eab-b096-a68bb961c856 service nova] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:24:15.435188 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff7a81b-f401-4eab-b096-a68bb961c856 service nova] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:24:15.435472 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff7a81b-f401-4eab-b096-a68bb961c856 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:24:15.444325 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff7a81b-f401-4eab-b096-a68bb961c856 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:24:15.445903 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff7a81b-f401-4eab-b096-a68bb961c856 service nova] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:24:15.456168 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 683/1368] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Jun 15 10:24:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:24:17.677449 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7df46e3-0b0c-48aa-bf3c-431ec36cb3d5 None None] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:24:17.678024 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7df46e3-0b0c-48aa-bf3c-431ec36cb3d5 None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:24:17.678024 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7df46e3-0b0c-48aa-bf3c-431ec36cb3d5 None None] PATH_INFO: `/` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:24:17.679410 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 686/1369] 10.222.0.49 () {54 vars in 926 bytes} [Wed Jun 15 10:24:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:24:17.684816 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753b42a7-3135-4aae-abba-53bf539e0c6e None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:24:17.687351 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753b42a7-3135-4aae-abba-53bf539e0c6e None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:24:17.687739 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753b42a7-3135-4aae-abba-53bf539e0c6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:24:17.707920 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-753b42a7-3135-4aae-abba-53bf539e0c6e None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:24:17.709271 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-753b42a7-3135-4aae-abba-53bf539e0c6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:24:17.717755 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 684/1370] 10.222.0.49 () {58 vars in 1023 bytes} [Wed Jun 15 10:24:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:24:17.762614 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7df46e3-0b0c-48aa-bf3c-431ec36cb3d5 None None] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:24:17.783484 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7df46e3-0b0c-48aa-bf3c-431ec36cb3d5 None None] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:24:17.791853 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0141e185-f5f5-4e77-a820-43387ab0545c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ac54528370d4747b6e8777fba9d0c82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '559b3e713f5844b6ae37151d56dd419f', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:24:17.793022 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0141e185-f5f5-4e77-a820-43387ab0545c service nova] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:24:17.793524 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0141e185-f5f5-4e77-a820-43387ab0545c service nova] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:24:17.794025 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0141e185-f5f5-4e77-a820-43387ab0545c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:24:17.804814 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0141e185-f5f5-4e77-a820-43387ab0545c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:24:17.806660 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0141e185-f5f5-4e77-a820-43387ab0545c service nova] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:24:17.817108 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 687/1371] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Jun 15 10:24:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:24:18.241280 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc667c33-48de-4bfe-9d6b-3fac99e71e29 None None] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:24:18.241483 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc667c33-48de-4bfe-9d6b-3fac99e71e29 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:24:18.241739 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc667c33-48de-4bfe-9d6b-3fac99e71e29 None None] PATH_INFO: `/` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:24:18.243480 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 685/1372] 10.222.0.49 () {54 vars in 926 bytes} [Wed Jun 15 10:24:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:24:18.248626 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f7d14c-f716-461e-b70d-e8b8879b9897 None None] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:24:18.248917 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f7d14c-f716-461e-b70d-e8b8879b9897 None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:24:18.249194 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f7d14c-f716-461e-b70d-e8b8879b9897 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:24:18.267880 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.auth.core [None req-f8f7d14c-f716-461e-b70d-e8b8879b9897 None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:24:18.269079 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-f8f7d14c-f716-461e-b70d-e8b8879b9897 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:24:18.277155 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 688/1373] 10.222.0.49 () {58 vars in 1023 bytes} [Wed Jun 15 10:24:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:24:18.541713 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a48ecc2-9cd4-4aa1-9491-2eab30c19c49 None None] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:24:18.542001 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a48ecc2-9cd4-4aa1-9491-2eab30c19c49 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:24:18.542280 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a48ecc2-9cd4-4aa1-9491-2eab30c19c49 None None] PATH_INFO: `/` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:24:18.544073 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 686/1374] 10.222.0.49 () {54 vars in 963 bytes} [Wed Jun 15 10:24:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:24:18.549239 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cdae87-055b-41f4-8428-2a26d8bba2c3 None None] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:24:18.549538 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cdae87-055b-41f4-8428-2a26d8bba2c3 None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:24:18.549820 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cdae87-055b-41f4-8428-2a26d8bba2c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:24:18.557030 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=100 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:24:18.607186 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.auth.core [None req-a1cdae87-055b-41f4-8428-2a26d8bba2c3 None None] MFA Rules not processed for user `20d1327553404b80aecee87c78fcb52b`. Rule list: `[]` (Enabled: `True`). {{(pid=80119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:24:18.608409 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-a1cdae87-055b-41f4-8428-2a26d8bba2c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:24:18.678201 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 689/1375] 10.222.0.49 () {58 vars in 1060 bytes} [Wed Jun 15 10:24:18 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:24:18.685603 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a48ecc2-9cd4-4aa1-9491-2eab30c19c49 None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:24:18.697042 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a48ecc2-9cd4-4aa1-9491-2eab30c19c49 None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:24:18.700561 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883d4e94-a2d2-4822-9a86-0ad0f0f275c9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20d1327553404b80aecee87c78fcb52b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '559b3e713f5844b6ae37151d56dd419f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:24:18.701235 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883d4e94-a2d2-4822-9a86-0ad0f0f275c9 service cinder] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:24:18.701523 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883d4e94-a2d2-4822-9a86-0ad0f0f275c9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:24:18.701803 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883d4e94-a2d2-4822-9a86-0ad0f0f275c9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:24:18.711864 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-883d4e94-a2d2-4822-9a86-0ad0f0f275c9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:24:18.713296 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-883d4e94-a2d2-4822-9a86-0ad0f0f275c9 service cinder] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:24:18.723031 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 687/1376] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Jun 15 10:24:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:24:19.995522 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7857cfc5-7cef-4401-af89-07ce7777f03a None None] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:24:19.996059 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7857cfc5-7cef-4401-af89-07ce7777f03a None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:24:19.996178 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7857cfc5-7cef-4401-af89-07ce7777f03a None None] PATH_INFO: `/` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:24:19.997484 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 690/1377] 10.222.0.49 () {54 vars in 926 bytes} [Wed Jun 15 10:24:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:24:20.003260 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53246a0b-7dd7-4b1b-bc54-57e3e2a5cedb None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:24:20.003684 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53246a0b-7dd7-4b1b-bc54-57e3e2a5cedb None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:24:20.003829 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53246a0b-7dd7-4b1b-bc54-57e3e2a5cedb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:24:20.021082 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-53246a0b-7dd7-4b1b-bc54-57e3e2a5cedb None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:24:20.022286 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-53246a0b-7dd7-4b1b-bc54-57e3e2a5cedb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:24:20.030375 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 688/1378] 10.222.0.49 () {58 vars in 1023 bytes} [Wed Jun 15 10:24:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:24:20.074724 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7857cfc5-7cef-4401-af89-07ce7777f03a None None] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:24:20.088991 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7857cfc5-7cef-4401-af89-07ce7777f03a None None] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:24:20.094499 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb45f79-d426-4c8c-9783-82de2f24651b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ac54528370d4747b6e8777fba9d0c82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '559b3e713f5844b6ae37151d56dd419f', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:24:20.095159 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb45f79-d426-4c8c-9783-82de2f24651b service nova] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:24:20.095465 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb45f79-d426-4c8c-9783-82de2f24651b service nova] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:24:20.095741 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb45f79-d426-4c8c-9783-82de2f24651b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:24:20.103923 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb45f79-d426-4c8c-9783-82de2f24651b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:24:20.105496 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb45f79-d426-4c8c-9783-82de2f24651b service nova] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:24:20.115623 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 691/1379] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Jun 15 10:24:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:24:22.273545 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4962830c-727d-4a21-a13d-880966f74dc3 None None] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:24:22.274400 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4962830c-727d-4a21-a13d-880966f74dc3 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:24:22.274836 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4962830c-727d-4a21-a13d-880966f74dc3 None None] PATH_INFO: `/` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:24:22.276438 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 689/1380] 10.222.0.49 () {54 vars in 926 bytes} [Wed Jun 15 10:24:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:24:22.282093 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81aff4be-d52c-4a0c-b5d9-d02580c54ff1 None None] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:24:22.282913 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81aff4be-d52c-4a0c-b5d9-d02580c54ff1 None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:24:22.283225 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81aff4be-d52c-4a0c-b5d9-d02580c54ff1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:24:22.309068 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.auth.core [None req-81aff4be-d52c-4a0c-b5d9-d02580c54ff1 None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:24:22.310227 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-81aff4be-d52c-4a0c-b5d9-d02580c54ff1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:24:22.322505 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 692/1381] 10.222.0.49 () {58 vars in 1023 bytes} [Wed Jun 15 10:24:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:24:22.369157 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4962830c-727d-4a21-a13d-880966f74dc3 None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:24:22.390568 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4962830c-727d-4a21-a13d-880966f74dc3 None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:24:22.396565 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81776b61-6ade-4a25-98a7-62221d27f24a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ac54528370d4747b6e8777fba9d0c82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '559b3e713f5844b6ae37151d56dd419f', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:24:22.397328 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81776b61-6ade-4a25-98a7-62221d27f24a service nova] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:24:22.397829 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81776b61-6ade-4a25-98a7-62221d27f24a service nova] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:24:22.398228 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81776b61-6ade-4a25-98a7-62221d27f24a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:24:22.407270 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81776b61-6ade-4a25-98a7-62221d27f24a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:24:22.409047 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81776b61-6ade-4a25-98a7-62221d27f24a service nova] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:24:22.419155 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 690/1382] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Jun 15 10:24:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:24:24.597718 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b231dc-9fc4-48f7-a8bf-e2b8a28eb549 None None] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:24:24.598592 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b231dc-9fc4-48f7-a8bf-e2b8a28eb549 None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:24:24.598943 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b231dc-9fc4-48f7-a8bf-e2b8a28eb549 None None] PATH_INFO: `/` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:24:24.600546 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 693/1383] 10.222.0.49 () {54 vars in 926 bytes} [Wed Jun 15 10:24:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:24:24.607157 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b678a2db-a48f-41d2-a391-a2f0c3631168 None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:24:24.607561 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b678a2db-a48f-41d2-a391-a2f0c3631168 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:24:24.607716 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b678a2db-a48f-41d2-a391-a2f0c3631168 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:24:24.628039 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-b678a2db-a48f-41d2-a391-a2f0c3631168 None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:24:24.629228 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-b678a2db-a48f-41d2-a391-a2f0c3631168 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:24:24.637296 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 691/1384] 10.222.0.49 () {58 vars in 1023 bytes} [Wed Jun 15 10:24:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:24:24.679281 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b231dc-9fc4-48f7-a8bf-e2b8a28eb549 None None] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:24:24.695851 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b231dc-9fc4-48f7-a8bf-e2b8a28eb549 None None] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:24:24.702019 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34d2124-c89f-412e-b263-40e1b3ef7bdc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ac54528370d4747b6e8777fba9d0c82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '559b3e713f5844b6ae37151d56dd419f', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:24:24.703488 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34d2124-c89f-412e-b263-40e1b3ef7bdc service nova] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:24:24.703888 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34d2124-c89f-412e-b263-40e1b3ef7bdc service nova] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:24:24.704275 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34d2124-c89f-412e-b263-40e1b3ef7bdc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:24:24.714698 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34d2124-c89f-412e-b263-40e1b3ef7bdc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:24:24.716473 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34d2124-c89f-412e-b263-40e1b3ef7bdc service nova] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:24:24.728124 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 694/1385] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Jun 15 10:24:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:24:26.978645 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debdfd12-a044-4de3-894b-74b157797547 None None] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:24:26.979337 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debdfd12-a044-4de3-894b-74b157797547 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:24:26.979337 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debdfd12-a044-4de3-894b-74b157797547 None None] PATH_INFO: `/` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:24:26.980658 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 692/1386] 10.222.0.49 () {54 vars in 926 bytes} [Wed Jun 15 10:24:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:24:26.987535 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcaa99d7-84ca-4c67-a252-34808454e830 None None] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:24:26.987971 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcaa99d7-84ca-4c67-a252-34808454e830 None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:24:26.988308 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcaa99d7-84ca-4c67-a252-34808454e830 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:24:27.010589 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.auth.core [None req-bcaa99d7-84ca-4c67-a252-34808454e830 None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:24:27.012225 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-bcaa99d7-84ca-4c67-a252-34808454e830 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:24:27.022039 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 695/1387] 10.222.0.49 () {58 vars in 1023 bytes} [Wed Jun 15 10:24:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:24:27.074291 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debdfd12-a044-4de3-894b-74b157797547 None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:24:27.088823 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debdfd12-a044-4de3-894b-74b157797547 None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:24:27.094842 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5911682c-4999-4fbd-9da7-3e3cd9523c78 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ac54528370d4747b6e8777fba9d0c82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '559b3e713f5844b6ae37151d56dd419f', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:24:27.095510 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5911682c-4999-4fbd-9da7-3e3cd9523c78 service nova] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:24:27.095790 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5911682c-4999-4fbd-9da7-3e3cd9523c78 service nova] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:24:27.096063 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5911682c-4999-4fbd-9da7-3e3cd9523c78 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:24:27.104465 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5911682c-4999-4fbd-9da7-3e3cd9523c78 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:24:27.106022 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5911682c-4999-4fbd-9da7-3e3cd9523c78 service nova] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:24:27.116205 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 693/1388] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Jun 15 10:24:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:24:29.335514 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9228117a-d1c8-4cc2-a0a5-775494f2c0c6 None None] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:24:29.336190 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9228117a-d1c8-4cc2-a0a5-775494f2c0c6 None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:24:29.336190 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9228117a-d1c8-4cc2-a0a5-775494f2c0c6 None None] PATH_INFO: `/` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:24:29.337420 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 696/1389] 10.222.0.49 () {54 vars in 926 bytes} [Wed Jun 15 10:24:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:24:29.342497 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4c2027-b19d-485c-9f85-8e8c3fc14d7a None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:24:29.343440 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4c2027-b19d-485c-9f85-8e8c3fc14d7a None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:24:29.343616 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4c2027-b19d-485c-9f85-8e8c3fc14d7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:24:29.361998 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-0b4c2027-b19d-485c-9f85-8e8c3fc14d7a None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:24:29.363359 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-0b4c2027-b19d-485c-9f85-8e8c3fc14d7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:24:29.371941 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 694/1390] 10.222.0.49 () {58 vars in 1023 bytes} [Wed Jun 15 10:24:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:24:29.420855 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9228117a-d1c8-4cc2-a0a5-775494f2c0c6 None None] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:24:29.440119 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9228117a-d1c8-4cc2-a0a5-775494f2c0c6 None None] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:24:29.446989 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263a118c-e4f7-4f8b-8848-0386b559e8c3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ac54528370d4747b6e8777fba9d0c82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '559b3e713f5844b6ae37151d56dd419f', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:24:29.447894 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263a118c-e4f7-4f8b-8848-0386b559e8c3 service nova] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:24:29.448265 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263a118c-e4f7-4f8b-8848-0386b559e8c3 service nova] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:24:29.448618 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263a118c-e4f7-4f8b-8848-0386b559e8c3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:24:29.459017 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-263a118c-e4f7-4f8b-8848-0386b559e8c3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:24:29.461112 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-263a118c-e4f7-4f8b-8848-0386b559e8c3 service nova] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:24:29.473943 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 697/1391] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Jun 15 10:24:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:24:32.220386 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9136a4-c63b-4e42-a7b6-299f28576725 None None] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:24:32.220974 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9136a4-c63b-4e42-a7b6-299f28576725 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:24:32.220974 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9136a4-c63b-4e42-a7b6-299f28576725 None None] PATH_INFO: `/` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:24:32.222724 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 695/1392] 10.222.0.49 () {54 vars in 926 bytes} [Wed Jun 15 10:24:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:24:32.227560 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c67920-808d-42c8-b715-b1cb8bd1384c None None] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:24:32.228027 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c67920-808d-42c8-b715-b1cb8bd1384c None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:24:32.228139 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c67920-808d-42c8-b715-b1cb8bd1384c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:24:32.246408 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.auth.core [None req-59c67920-808d-42c8-b715-b1cb8bd1384c None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:24:32.247662 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-59c67920-808d-42c8-b715-b1cb8bd1384c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:24:32.256356 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 698/1393] 10.222.0.49 () {58 vars in 1023 bytes} [Wed Jun 15 10:24:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:24:32.299707 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9136a4-c63b-4e42-a7b6-299f28576725 None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:24:32.317199 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9136a4-c63b-4e42-a7b6-299f28576725 None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:24:32.322769 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe26c03f-3504-4892-bd5b-b49fd0df8300 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ac54528370d4747b6e8777fba9d0c82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '559b3e713f5844b6ae37151d56dd419f', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:24:32.323447 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe26c03f-3504-4892-bd5b-b49fd0df8300 service nova] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:24:32.323728 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe26c03f-3504-4892-bd5b-b49fd0df8300 service nova] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:24:32.324006 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe26c03f-3504-4892-bd5b-b49fd0df8300 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:24:32.332671 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe26c03f-3504-4892-bd5b-b49fd0df8300 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:24:32.334255 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe26c03f-3504-4892-bd5b-b49fd0df8300 service nova] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:24:32.344536 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 696/1394] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Jun 15 10:24:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:24:34.261911 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b10019-5f2b-40af-9231-7b40f26e4171 None None] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:24:34.262484 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b10019-5f2b-40af-9231-7b40f26e4171 None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:24:34.262484 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b10019-5f2b-40af-9231-7b40f26e4171 None None] PATH_INFO: `/` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:24:34.263905 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 699/1395] 10.222.0.49 () {54 vars in 926 bytes} [Wed Jun 15 10:24:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:24:34.269670 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f78864-033e-4746-be24-7d8508ba1e3e None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:24:34.270098 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f78864-033e-4746-be24-7d8508ba1e3e None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:24:34.270360 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f78864-033e-4746-be24-7d8508ba1e3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:24:34.288463 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-01f78864-033e-4746-be24-7d8508ba1e3e None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:24:34.289649 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-01f78864-033e-4746-be24-7d8508ba1e3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:24:34.297703 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 697/1396] 10.222.0.49 () {58 vars in 1023 bytes} [Wed Jun 15 10:24:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:24:34.353091 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0678e2a-1c0f-41d0-8ccf-fe21fc637347 None None] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:24:34.353384 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0678e2a-1c0f-41d0-8ccf-fe21fc637347 None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:24:34.353664 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0678e2a-1c0f-41d0-8ccf-fe21fc637347 None None] PATH_INFO: `/` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:24:34.355195 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 700/1397] 10.222.0.49 () {54 vars in 962 bytes} [Wed Jun 15 10:24:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:24:34.360116 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56df795-763f-4760-aece-5078148fa1d3 None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:24:34.360413 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56df795-763f-4760-aece-5078148fa1d3 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:24:34.360693 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56df795-763f-4760-aece-5078148fa1d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:24:34.409661 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-a56df795-763f-4760-aece-5078148fa1d3 None None] MFA Rules not processed for user `91f2fe3be62e49548eae208e57bf2a93`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:24:34.410873 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-a56df795-763f-4760-aece-5078148fa1d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:24:34.475896 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 698/1398] 10.222.0.49 () {58 vars in 1059 bytes} [Wed Jun 15 10:24:34 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:24:34.483773 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0678e2a-1c0f-41d0-8ccf-fe21fc637347 None None] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:24:34.492491 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1365f38e-8da8-4bea-af86-43eaaf2ad877 None None] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:24:34.492780 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1365f38e-8da8-4bea-af86-43eaaf2ad877 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:24:34.493077 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1365f38e-8da8-4bea-af86-43eaaf2ad877 None None] PATH_INFO: `/` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:24:34.494419 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 699/1399] 10.222.0.49 () {54 vars in 926 bytes} [Wed Jun 15 10:24:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 15 10:24:34.494790 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0678e2a-1c0f-41d0-8ccf-fe21fc637347 None None] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:24:34.498839 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c12611-3bc9-43f8-a28b-c20213160b00 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91f2fe3be62e49548eae208e57bf2a93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '559b3e713f5844b6ae37151d56dd419f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:24:34.499401 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8505c59-31ef-439a-bb3e-98e5337bc183 None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:24:34.499686 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8505c59-31ef-439a-bb3e-98e5337bc183 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:24:34.499811 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c12611-3bc9-43f8-a28b-c20213160b00 service glance] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:24:34.499963 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8505c59-31ef-439a-bb3e-98e5337bc183 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:24:34.500202 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c12611-3bc9-43f8-a28b-c20213160b00 service glance] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:24:34.500494 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c12611-3bc9-43f8-a28b-c20213160b00 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:24:34.509792 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c12611-3bc9-43f8-a28b-c20213160b00 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:24:34.511301 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c12611-3bc9-43f8-a28b-c20213160b00 service glance] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:24:34.517617 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-a8505c59-31ef-439a-bb3e-98e5337bc183 None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:24:34.518751 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-a8505c59-31ef-439a-bb3e-98e5337bc183 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:24:34.521929 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 701/1400] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Jun 15 10:24:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:24:34.527950 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 700/1401] 10.222.0.49 () {58 vars in 1023 bytes} [Wed Jun 15 10:24:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 10:24:34.574641 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c12611-3bc9-43f8-a28b-c20213160b00 service glance] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:24:34.587582 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c12611-3bc9-43f8-a28b-c20213160b00 service glance] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:24:34.593255 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0738b1a-ab27-4fb0-9c37-8b6502388ad4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ac54528370d4747b6e8777fba9d0c82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '559b3e713f5844b6ae37151d56dd419f', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:24:34.593938 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0738b1a-ab27-4fb0-9c37-8b6502388ad4 service nova] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:24:34.594221 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0738b1a-ab27-4fb0-9c37-8b6502388ad4 service nova] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:24:34.594497 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0738b1a-ab27-4fb0-9c37-8b6502388ad4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:24:34.604279 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0738b1a-ab27-4fb0-9c37-8b6502388ad4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:24:34.605910 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0738b1a-ab27-4fb0-9c37-8b6502388ad4 service nova] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:24:34.616829 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 702/1402] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Jun 15 10:24:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:24:35.238689 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e846b0-216a-4373-92dd-77f8509c3a60 None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:24:35.239055 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e846b0-216a-4373-92dd-77f8509c3a60 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:24:35.239432 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e846b0-216a-4373-92dd-77f8509c3a60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:24:35.246598 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=97 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:24:35.288510 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-85e846b0-216a-4373-92dd-77f8509c3a60 None None] MFA Rules not processed for user `103e10244eca444b86b9c51ff314d445`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:24:35.289984 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-85e846b0-216a-4373-92dd-77f8509c3a60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:24:35.354437 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 701/1403] 10.222.0.49 () {58 vars in 1009 bytes} [Wed Jun 15 10:24:35 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:24:35.398294 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10acc8f4-d7f6-484e-9555-9687d41450e8 None None] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:24:35.398780 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10acc8f4-d7f6-484e-9555-9687d41450e8 None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:24:35.398995 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10acc8f4-d7f6-484e-9555-9687d41450e8 None None] PATH_INFO: `/` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:24:35.400560 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 703/1404] 10.222.0.49 () {54 vars in 942 bytes} [Wed Jun 15 10:24:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:24:35.407515 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471760af-40a1-472e-84d3-e18356ef5993 None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:24:35.407898 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471760af-40a1-472e-84d3-e18356ef5993 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:24:35.408270 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471760af-40a1-472e-84d3-e18356ef5993 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:24:35.455949 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-471760af-40a1-472e-84d3-e18356ef5993 None None] MFA Rules not processed for user `3a69f99210444f9696e8b22615fb4a41`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:24:35.457170 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-471760af-40a1-472e-84d3-e18356ef5993 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:24:35.526360 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 702/1405] 10.222.0.49 () {58 vars in 1039 bytes} [Wed Jun 15 10:24:35 2022] POST /identity/v3/auth/tokens => generated 3294 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:24:35.536443 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10acc8f4-d7f6-484e-9555-9687d41450e8 None None] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:24:35.550314 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10acc8f4-d7f6-484e-9555-9687d41450e8 None None] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:24:35.555567 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928a78f3-d927-4be3-b5ac-f12193681df4 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a69f99210444f9696e8b22615fb4a41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '559b3e713f5844b6ae37151d56dd419f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:24:35.556290 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928a78f3-d927-4be3-b5ac-f12193681df4 service swift] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:24:35.556638 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928a78f3-d927-4be3-b5ac-f12193681df4 service swift] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:24:35.557075 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928a78f3-d927-4be3-b5ac-f12193681df4 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:24:35.565671 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-928a78f3-d927-4be3-b5ac-f12193681df4 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:24:35.567297 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-928a78f3-d927-4be3-b5ac-f12193681df4 service swift] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:24:35.575212 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 704/1406] 10.222.0.49 () {60 vars in 1384 bytes} [Wed Jun 15 10:24:35 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 15 10:24:36.797189 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87316fc5-712c-4ca9-bc08-b4de6945c00d None None] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:24:36.797592 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87316fc5-712c-4ca9-bc08-b4de6945c00d None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:24:36.797787 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87316fc5-712c-4ca9-bc08-b4de6945c00d None None] PATH_INFO: `/` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:24:36.799306 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 703/1407] 10.222.0.49 () {54 vars in 926 bytes} [Wed Jun 15 10:24:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:24:36.804611 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342181c6-81c7-44cc-a40a-ce29eaf516e3 None None] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:24:36.805115 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342181c6-81c7-44cc-a40a-ce29eaf516e3 None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:24:36.805233 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342181c6-81c7-44cc-a40a-ce29eaf516e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:24:36.823783 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.auth.core [None req-342181c6-81c7-44cc-a40a-ce29eaf516e3 None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:24:36.825279 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-342181c6-81c7-44cc-a40a-ce29eaf516e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:24:36.834063 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 705/1408] 10.222.0.49 () {58 vars in 1023 bytes} [Wed Jun 15 10:24:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:24:36.878043 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87316fc5-712c-4ca9-bc08-b4de6945c00d None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:24:36.892780 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87316fc5-712c-4ca9-bc08-b4de6945c00d None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:24:36.899440 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8044fb78-5784-4bf9-8d4d-41ba7cc3c082 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ac54528370d4747b6e8777fba9d0c82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '559b3e713f5844b6ae37151d56dd419f', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:24:36.900097 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8044fb78-5784-4bf9-8d4d-41ba7cc3c082 service nova] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:24:36.900505 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8044fb78-5784-4bf9-8d4d-41ba7cc3c082 service nova] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:24:36.900795 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8044fb78-5784-4bf9-8d4d-41ba7cc3c082 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:24:36.909953 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8044fb78-5784-4bf9-8d4d-41ba7cc3c082 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:24:36.911698 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8044fb78-5784-4bf9-8d4d-41ba7cc3c082 service nova] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:24:36.922080 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 704/1409] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Jun 15 10:24:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:24:46.818549 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=84 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:24:46.917219 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=88 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:24:57.917988 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bacdf5f-5c22-4f19-9c07-e128fe23dc3a None None] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:24:57.918472 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bacdf5f-5c22-4f19-9c07-e128fe23dc3a None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:24:57.918703 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bacdf5f-5c22-4f19-9c07-e128fe23dc3a None None] PATH_INFO: `/` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:24:57.920616 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 706/1410] 10.222.0.49 () {54 vars in 918 bytes} [Wed Jun 15 10:24:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:24:57.925910 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfa9e76-6e16-4a8b-9348-f46d739eb815 None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:24:57.926366 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfa9e76-6e16-4a8b-9348-f46d739eb815 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:24:57.926468 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfa9e76-6e16-4a8b-9348-f46d739eb815 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:24:57.944185 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-0bfa9e76-6e16-4a8b-9348-f46d739eb815 None None] MFA Rules not processed for user `dd03612a525644518517069b3fa43522`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:24:57.945397 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-0bfa9e76-6e16-4a8b-9348-f46d739eb815 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:24:57.953921 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 705/1411] 10.222.0.49 () {58 vars in 1015 bytes} [Wed Jun 15 10:24:57 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:24:57.968499 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a541384-5470-4472-aff7-54503a015f71 None None] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:24:57.968790 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a541384-5470-4472-aff7-54503a015f71 None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:24:57.969067 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a541384-5470-4472-aff7-54503a015f71 None None] PATH_INFO: `/` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:24:57.970407 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 707/1412] 10.222.0.49 () {54 vars in 918 bytes} [Wed Jun 15 10:24:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:24:57.974917 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d7f899-975f-4306-802e-914e8ed884db None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:24:57.975255 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d7f899-975f-4306-802e-914e8ed884db None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:24:57.975541 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d7f899-975f-4306-802e-914e8ed884db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:24:57.993430 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-71d7f899-975f-4306-802e-914e8ed884db None None] MFA Rules not processed for user `5ac54528370d4747b6e8777fba9d0c82`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:24:57.994633 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-71d7f899-975f-4306-802e-914e8ed884db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:24:58.003727 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 706/1413] 10.222.0.49 () {58 vars in 1015 bytes} [Wed Jun 15 10:24:57 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 10:25:00.596881 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593bc2f1-1217-4582-a63c-f6fa258ace36 None None] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:25:00.597428 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593bc2f1-1217-4582-a63c-f6fa258ace36 None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:25:00.597428 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593bc2f1-1217-4582-a63c-f6fa258ace36 None None] PATH_INFO: `/` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:25:00.598792 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 708/1414] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jun 15 10:25:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:25:00.604145 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5d9f35-a6f5-482a-b736-86c81ba5d3ef None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:25:00.604624 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5d9f35-a6f5-482a-b736-86c81ba5d3ef None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:25:00.604739 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5d9f35-a6f5-482a-b736-86c81ba5d3ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:25:00.622172 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-cb5d9f35-a6f5-482a-b736-86c81ba5d3ef None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:25:00.623451 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-cb5d9f35-a6f5-482a-b736-86c81ba5d3ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:25:00.631258 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 707/1415] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jun 15 10:25:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:25:00.674403 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593bc2f1-1217-4582-a63c-f6fa258ace36 None None] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:25:00.688389 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593bc2f1-1217-4582-a63c-f6fa258ace36 None None] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:25:00.693678 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76db1b9-3162-458d-a29f-472061b1be81 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ac54528370d4747b6e8777fba9d0c82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '559b3e713f5844b6ae37151d56dd419f', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:25:00.694305 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76db1b9-3162-458d-a29f-472061b1be81 service nova] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:25:00.694587 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76db1b9-3162-458d-a29f-472061b1be81 service nova] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:25:00.694859 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76db1b9-3162-458d-a29f-472061b1be81 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:25:00.703372 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e76db1b9-3162-458d-a29f-472061b1be81 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:25:00.704935 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e76db1b9-3162-458d-a29f-472061b1be81 service nova] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:25:00.714072 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 709/1416] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Jun 15 10:25:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 10:25:01.664153 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93b5c7a-1499-4a08-b441-5d2cf2e865ac None None] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:25:01.664611 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93b5c7a-1499-4a08-b441-5d2cf2e865ac None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:25:01.664736 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93b5c7a-1499-4a08-b441-5d2cf2e865ac None None] PATH_INFO: `/` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:25:01.666101 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 708/1417] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jun 15 10:25:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:25:01.671699 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020c5f37-f9eb-4a7c-b475-8042f24a1cc4 None None] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:25:01.672143 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020c5f37-f9eb-4a7c-b475-8042f24a1cc4 None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:25:01.672254 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020c5f37-f9eb-4a7c-b475-8042f24a1cc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:25:01.690031 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.auth.core [None req-020c5f37-f9eb-4a7c-b475-8042f24a1cc4 None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:25:01.691350 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-020c5f37-f9eb-4a7c-b475-8042f24a1cc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:25:01.699110 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 710/1418] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jun 15 10:25:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:25:01.851003 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93b5c7a-1499-4a08-b441-5d2cf2e865ac None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:25:01.865903 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93b5c7a-1499-4a08-b441-5d2cf2e865ac None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:25:01.871136 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab741c9-c106-4f93-b81d-3468041082a9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ac54528370d4747b6e8777fba9d0c82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '559b3e713f5844b6ae37151d56dd419f', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:25:01.871766 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab741c9-c106-4f93-b81d-3468041082a9 service nova] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:25:01.872047 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab741c9-c106-4f93-b81d-3468041082a9 service nova] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:25:01.872321 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab741c9-c106-4f93-b81d-3468041082a9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:25:01.880220 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab741c9-c106-4f93-b81d-3468041082a9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:25:01.881784 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab741c9-c106-4f93-b81d-3468041082a9 service nova] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:25:01.891713 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 709/1419] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Jun 15 10:25:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:25:02.820860 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5196c9-d7c9-4d78-8228-0adaed162455 None None] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:25:02.821289 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5196c9-d7c9-4d78-8228-0adaed162455 None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:25:02.821499 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5196c9-d7c9-4d78-8228-0adaed162455 None None] PATH_INFO: `/` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:25:02.822885 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 711/1420] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jun 15 10:25:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:25:02.827870 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c0cb8a-56c9-4c06-a96e-dc97d7044f6d None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:25:02.828318 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c0cb8a-56c9-4c06-a96e-dc97d7044f6d None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:25:02.828426 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c0cb8a-56c9-4c06-a96e-dc97d7044f6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:25:02.846866 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-73c0cb8a-56c9-4c06-a96e-dc97d7044f6d None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:25:02.848094 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-73c0cb8a-56c9-4c06-a96e-dc97d7044f6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:25:02.855710 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 710/1421] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jun 15 10:25:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:25:03.006495 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5196c9-d7c9-4d78-8228-0adaed162455 None None] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:25:03.020591 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5196c9-d7c9-4d78-8228-0adaed162455 None None] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:25:03.025947 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62fd7fb3-fbc8-47a1-89eb-1d5bb41b011b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ac54528370d4747b6e8777fba9d0c82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '559b3e713f5844b6ae37151d56dd419f', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:25:03.026585 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fd7fb3-fbc8-47a1-89eb-1d5bb41b011b service nova] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:25:03.026874 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fd7fb3-fbc8-47a1-89eb-1d5bb41b011b service nova] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:25:03.027188 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fd7fb3-fbc8-47a1-89eb-1d5bb41b011b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:25:03.035342 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62fd7fb3-fbc8-47a1-89eb-1d5bb41b011b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:25:03.037171 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62fd7fb3-fbc8-47a1-89eb-1d5bb41b011b service nova] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:25:03.047298 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 712/1422] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Jun 15 10:25:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:25:03.949462 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c022d5dc-95de-4c5d-b1b2-7b28d943aaa6 None None] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:25:03.949999 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c022d5dc-95de-4c5d-b1b2-7b28d943aaa6 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:25:03.949999 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c022d5dc-95de-4c5d-b1b2-7b28d943aaa6 None None] PATH_INFO: `/` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:25:03.951945 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 711/1423] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jun 15 10:25:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:25:03.956767 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b57e292-5ccd-4aed-b864-76ee4666835e None None] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:25:03.957165 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b57e292-5ccd-4aed-b864-76ee4666835e None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:25:03.957336 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b57e292-5ccd-4aed-b864-76ee4666835e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:25:03.974893 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.auth.core [None req-9b57e292-5ccd-4aed-b864-76ee4666835e None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:25:03.976139 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-9b57e292-5ccd-4aed-b864-76ee4666835e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:25:03.983967 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 713/1424] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jun 15 10:25:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:25:04.136178 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c022d5dc-95de-4c5d-b1b2-7b28d943aaa6 None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:25:04.150896 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c022d5dc-95de-4c5d-b1b2-7b28d943aaa6 None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:25:04.156195 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7189eb8-cc8c-4fac-b008-e82f04087b6e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ac54528370d4747b6e8777fba9d0c82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '559b3e713f5844b6ae37151d56dd419f', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:25:04.156823 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7189eb8-cc8c-4fac-b008-e82f04087b6e service nova] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:25:04.157102 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7189eb8-cc8c-4fac-b008-e82f04087b6e service nova] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:25:04.157370 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7189eb8-cc8c-4fac-b008-e82f04087b6e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:25:04.165241 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7189eb8-cc8c-4fac-b008-e82f04087b6e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:25:04.167063 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7189eb8-cc8c-4fac-b008-e82f04087b6e service nova] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:25:04.176290 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 712/1425] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Jun 15 10:25:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:25:05.072207 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06cdc1c-6313-4fed-87bf-1c00a28861ae None None] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:25:05.072767 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06cdc1c-6313-4fed-87bf-1c00a28861ae None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:25:05.072767 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06cdc1c-6313-4fed-87bf-1c00a28861ae None None] PATH_INFO: `/` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:25:05.074124 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 714/1426] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jun 15 10:25:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:25:05.079500 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1357b5d7-dfae-42bd-a5b8-7eb3a1afed76 None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:25:05.079936 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1357b5d7-dfae-42bd-a5b8-7eb3a1afed76 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:25:05.080041 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1357b5d7-dfae-42bd-a5b8-7eb3a1afed76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:25:05.098240 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-1357b5d7-dfae-42bd-a5b8-7eb3a1afed76 None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:25:05.099473 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-1357b5d7-dfae-42bd-a5b8-7eb3a1afed76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:25:05.107623 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 713/1427] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jun 15 10:25:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:25:05.257167 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06cdc1c-6313-4fed-87bf-1c00a28861ae None None] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:25:05.271902 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06cdc1c-6313-4fed-87bf-1c00a28861ae None None] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:25:05.277263 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258a3816-a26d-4955-a876-9416ed6bcc06 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ac54528370d4747b6e8777fba9d0c82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '559b3e713f5844b6ae37151d56dd419f', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:25:05.277902 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258a3816-a26d-4955-a876-9416ed6bcc06 service nova] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:25:05.278185 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258a3816-a26d-4955-a876-9416ed6bcc06 service nova] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:25:05.278462 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258a3816-a26d-4955-a876-9416ed6bcc06 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:25:05.287114 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-258a3816-a26d-4955-a876-9416ed6bcc06 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:25:05.288712 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-258a3816-a26d-4955-a876-9416ed6bcc06 service nova] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:25:05.298255 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 715/1428] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Jun 15 10:25:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:25:06.139052 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875102ff-bb17-4962-bb6f-e1f497389fdf None None] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:25:06.139633 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875102ff-bb17-4962-bb6f-e1f497389fdf None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:25:06.139754 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875102ff-bb17-4962-bb6f-e1f497389fdf None None] PATH_INFO: `/` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:25:06.141192 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 714/1429] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jun 15 10:25:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:25:06.146084 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98f2389-dc08-44a7-a041-552f695eb0d3 None None] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:25:06.146510 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98f2389-dc08-44a7-a041-552f695eb0d3 None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:25:06.146656 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98f2389-dc08-44a7-a041-552f695eb0d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:25:06.165124 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.auth.core [None req-e98f2389-dc08-44a7-a041-552f695eb0d3 None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:25:06.166651 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-e98f2389-dc08-44a7-a041-552f695eb0d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:25:06.175105 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 716/1430] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jun 15 10:25:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:25:06.325293 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875102ff-bb17-4962-bb6f-e1f497389fdf None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:25:06.339191 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875102ff-bb17-4962-bb6f-e1f497389fdf None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:25:06.345029 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67bb4a84-6713-49a6-b3c3-5e2286dda9aa service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ac54528370d4747b6e8777fba9d0c82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '559b3e713f5844b6ae37151d56dd419f', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:25:06.345658 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67bb4a84-6713-49a6-b3c3-5e2286dda9aa service nova] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:25:06.345939 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67bb4a84-6713-49a6-b3c3-5e2286dda9aa service nova] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:25:06.346212 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67bb4a84-6713-49a6-b3c3-5e2286dda9aa service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:25:06.354115 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67bb4a84-6713-49a6-b3c3-5e2286dda9aa service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:25:06.355897 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67bb4a84-6713-49a6-b3c3-5e2286dda9aa service nova] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:25:06.366435 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 715/1431] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Jun 15 10:25:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:25:07.222032 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460adda9-af27-4514-b46f-e9debc68d459 None None] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:25:07.222556 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460adda9-af27-4514-b46f-e9debc68d459 None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:25:07.222556 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460adda9-af27-4514-b46f-e9debc68d459 None None] PATH_INFO: `/` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:25:07.223952 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 717/1432] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jun 15 10:25:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:25:07.229098 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccee293-fa24-487c-a8e9-14d788eaf1bf None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:25:07.229509 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccee293-fa24-487c-a8e9-14d788eaf1bf None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:25:07.229649 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccee293-fa24-487c-a8e9-14d788eaf1bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:25:07.248481 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-1ccee293-fa24-487c-a8e9-14d788eaf1bf None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:25:07.249674 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-1ccee293-fa24-487c-a8e9-14d788eaf1bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:25:07.257356 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 716/1433] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jun 15 10:25:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:25:07.408209 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460adda9-af27-4514-b46f-e9debc68d459 None None] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:25:07.423457 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460adda9-af27-4514-b46f-e9debc68d459 None None] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:25:07.428774 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1803320-1cae-41a2-a408-f7108058ad8d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ac54528370d4747b6e8777fba9d0c82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '559b3e713f5844b6ae37151d56dd419f', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:25:07.429388 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1803320-1cae-41a2-a408-f7108058ad8d service nova] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:25:07.429670 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1803320-1cae-41a2-a408-f7108058ad8d service nova] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:25:07.429943 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1803320-1cae-41a2-a408-f7108058ad8d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:25:07.438497 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1803320-1cae-41a2-a408-f7108058ad8d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:25:07.440107 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1803320-1cae-41a2-a408-f7108058ad8d service nova] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:25:07.449229 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 718/1434] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Jun 15 10:25:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:25:08.300342 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e006b6-256f-4164-a51d-b61f221348a1 None None] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:25:08.300990 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e006b6-256f-4164-a51d-b61f221348a1 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:25:08.300990 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e006b6-256f-4164-a51d-b61f221348a1 None None] PATH_INFO: `/` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:25:08.302289 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 717/1435] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jun 15 10:25:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:25:08.307065 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acfa00d-3d26-46a5-93f0-c313d3aeb51e None None] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:25:08.307536 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acfa00d-3d26-46a5-93f0-c313d3aeb51e None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:25:08.307643 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acfa00d-3d26-46a5-93f0-c313d3aeb51e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:25:08.325549 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.auth.core [None req-2acfa00d-3d26-46a5-93f0-c313d3aeb51e None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:25:08.327328 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-2acfa00d-3d26-46a5-93f0-c313d3aeb51e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:25:08.334945 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 719/1436] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jun 15 10:25:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:25:08.484235 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e006b6-256f-4164-a51d-b61f221348a1 None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:25:08.499117 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e006b6-256f-4164-a51d-b61f221348a1 None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:25:08.504431 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44cf4b40-a5c1-4059-9eff-46f5db79161b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ac54528370d4747b6e8777fba9d0c82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '559b3e713f5844b6ae37151d56dd419f', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:25:08.505066 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cf4b40-a5c1-4059-9eff-46f5db79161b service nova] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:25:08.505348 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cf4b40-a5c1-4059-9eff-46f5db79161b service nova] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:25:08.505623 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cf4b40-a5c1-4059-9eff-46f5db79161b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:25:08.514164 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44cf4b40-a5c1-4059-9eff-46f5db79161b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:25:08.515748 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44cf4b40-a5c1-4059-9eff-46f5db79161b service nova] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:25:08.524992 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 718/1437] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Jun 15 10:25:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:25:09.377994 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f608c51f-15aa-4cb2-9bbe-76d064f0254b None None] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:25:09.378562 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f608c51f-15aa-4cb2-9bbe-76d064f0254b None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:25:09.378562 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f608c51f-15aa-4cb2-9bbe-76d064f0254b None None] PATH_INFO: `/` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:25:09.380012 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 720/1438] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jun 15 10:25:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:25:09.384806 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed450d5-4586-4b0b-abd1-8f7298a7df89 None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:25:09.385199 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed450d5-4586-4b0b-abd1-8f7298a7df89 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:25:09.385357 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed450d5-4586-4b0b-abd1-8f7298a7df89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:25:09.402835 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-3ed450d5-4586-4b0b-abd1-8f7298a7df89 None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:25:09.404585 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-3ed450d5-4586-4b0b-abd1-8f7298a7df89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:25:09.412330 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 719/1439] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jun 15 10:25:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:25:09.571231 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f608c51f-15aa-4cb2-9bbe-76d064f0254b None None] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:25:09.586037 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f608c51f-15aa-4cb2-9bbe-76d064f0254b None None] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:25:09.592031 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac5e43f-c86a-4e44-81fa-7dd08b70dcc3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ac54528370d4747b6e8777fba9d0c82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '559b3e713f5844b6ae37151d56dd419f', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:25:09.592671 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac5e43f-c86a-4e44-81fa-7dd08b70dcc3 service nova] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:25:09.592959 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac5e43f-c86a-4e44-81fa-7dd08b70dcc3 service nova] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:25:09.593240 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac5e43f-c86a-4e44-81fa-7dd08b70dcc3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:25:09.601166 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac5e43f-c86a-4e44-81fa-7dd08b70dcc3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:25:09.602742 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac5e43f-c86a-4e44-81fa-7dd08b70dcc3 service nova] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:25:09.611945 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 721/1440] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Jun 15 10:25:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:25:11.609486 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1e6ad2-90ab-4a41-8d7d-db6a00f0873e None None] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:25:11.610055 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1e6ad2-90ab-4a41-8d7d-db6a00f0873e None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:25:11.610055 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1e6ad2-90ab-4a41-8d7d-db6a00f0873e None None] PATH_INFO: `/` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:25:11.611548 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 720/1441] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jun 15 10:25:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:25:11.616944 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2501ada-83aa-4c15-b563-7119d9edd0f6 None None] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:25:11.617372 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2501ada-83aa-4c15-b563-7119d9edd0f6 None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:25:11.617497 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2501ada-83aa-4c15-b563-7119d9edd0f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:25:11.634828 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.auth.core [None req-e2501ada-83aa-4c15-b563-7119d9edd0f6 None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:25:11.636063 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-e2501ada-83aa-4c15-b563-7119d9edd0f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:25:11.644040 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 722/1442] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jun 15 10:25:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:25:11.682738 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1e6ad2-90ab-4a41-8d7d-db6a00f0873e None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:25:11.693112 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1e6ad2-90ab-4a41-8d7d-db6a00f0873e None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:25:11.696714 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e07745-8888-4e33-8674-fe269aa700fd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0cb44a17d205431793e9a8791e477a32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '559b3e713f5844b6ae37151d56dd419f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:25:11.697357 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e07745-8888-4e33-8674-fe269aa700fd service neutron] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:25:11.697665 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e07745-8888-4e33-8674-fe269aa700fd service neutron] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:25:11.697965 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e07745-8888-4e33-8674-fe269aa700fd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:25:11.707444 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e07745-8888-4e33-8674-fe269aa700fd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:25:11.708853 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e07745-8888-4e33-8674-fe269aa700fd service neutron] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:25:11.718827 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 721/1443] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Jun 15 10:25:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:25:14.060431 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27398bb-abc6-40fe-8f84-a39c7a6b8420 None None] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:25:14.060989 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27398bb-abc6-40fe-8f84-a39c7a6b8420 None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:25:14.060989 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27398bb-abc6-40fe-8f84-a39c7a6b8420 None None] PATH_INFO: `/` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:25:14.062345 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 723/1444] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jun 15 10:25:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:25:14.067595 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc54eac-636c-4ca1-9bfe-fe02b37df5b0 None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:25:14.068045 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc54eac-636c-4ca1-9bfe-fe02b37df5b0 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:25:14.068148 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc54eac-636c-4ca1-9bfe-fe02b37df5b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:25:14.086127 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-fbc54eac-636c-4ca1-9bfe-fe02b37df5b0 None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:25:14.087359 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-fbc54eac-636c-4ca1-9bfe-fe02b37df5b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:25:14.095830 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 722/1445] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jun 15 10:25:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:25:14.132568 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27398bb-abc6-40fe-8f84-a39c7a6b8420 None None] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:25:14.143597 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27398bb-abc6-40fe-8f84-a39c7a6b8420 None None] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:25:14.147232 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3b49f0-2dec-4ca1-a73a-d4aaea01302a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0cb44a17d205431793e9a8791e477a32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '559b3e713f5844b6ae37151d56dd419f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:25:14.147856 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3b49f0-2dec-4ca1-a73a-d4aaea01302a service neutron] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:25:14.148139 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3b49f0-2dec-4ca1-a73a-d4aaea01302a service neutron] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:25:14.148415 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3b49f0-2dec-4ca1-a73a-d4aaea01302a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:25:14.157715 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f3b49f0-2dec-4ca1-a73a-d4aaea01302a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:25:14.159169 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f3b49f0-2dec-4ca1-a73a-d4aaea01302a service neutron] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:25:14.168506 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 724/1446] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Jun 15 10:25:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 10:25:16.598628 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaddd8b1-e86c-40d1-883c-3151b14628a6 None None] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:25:16.599224 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaddd8b1-e86c-40d1-883c-3151b14628a6 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:25:16.599224 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaddd8b1-e86c-40d1-883c-3151b14628a6 None None] PATH_INFO: `/` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:25:16.600592 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 723/1447] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jun 15 10:25:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:25:16.605782 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c43bad-3bfa-42f3-bcad-4752ba135f37 None None] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:25:16.606209 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c43bad-3bfa-42f3-bcad-4752ba135f37 None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:25:16.606325 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c43bad-3bfa-42f3-bcad-4752ba135f37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:25:16.624808 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.auth.core [None req-19c43bad-3bfa-42f3-bcad-4752ba135f37 None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:25:16.625999 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-19c43bad-3bfa-42f3-bcad-4752ba135f37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:25:16.633628 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 725/1448] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jun 15 10:25:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:25:16.673116 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaddd8b1-e86c-40d1-883c-3151b14628a6 None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:25:16.683650 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaddd8b1-e86c-40d1-883c-3151b14628a6 None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:25:16.687285 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baaa3fba-26b6-47f0-9061-ebf1b5697994 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0cb44a17d205431793e9a8791e477a32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '559b3e713f5844b6ae37151d56dd419f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:25:16.687944 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baaa3fba-26b6-47f0-9061-ebf1b5697994 service neutron] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:25:16.688232 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baaa3fba-26b6-47f0-9061-ebf1b5697994 service neutron] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:25:16.688513 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baaa3fba-26b6-47f0-9061-ebf1b5697994 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:25:16.697512 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baaa3fba-26b6-47f0-9061-ebf1b5697994 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:25:16.698973 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baaa3fba-26b6-47f0-9061-ebf1b5697994 service neutron] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:25:16.708391 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 724/1449] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Jun 15 10:25:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:25:18.934528 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40718cc3-c016-4e2d-95de-b2e620210b97 None None] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:25:18.935077 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40718cc3-c016-4e2d-95de-b2e620210b97 None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:25:18.935077 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40718cc3-c016-4e2d-95de-b2e620210b97 None None] PATH_INFO: `/` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:25:18.936486 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 726/1450] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jun 15 10:25:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:25:18.941713 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e960925-f34c-4e15-860a-1488e5fe8833 None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:25:18.942106 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e960925-f34c-4e15-860a-1488e5fe8833 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:25:18.942269 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e960925-f34c-4e15-860a-1488e5fe8833 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:25:18.959421 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-8e960925-f34c-4e15-860a-1488e5fe8833 None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:25:18.960618 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-8e960925-f34c-4e15-860a-1488e5fe8833 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:25:18.968744 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 725/1451] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jun 15 10:25:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:25:19.006082 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40718cc3-c016-4e2d-95de-b2e620210b97 None None] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:25:19.016517 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40718cc3-c016-4e2d-95de-b2e620210b97 None None] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:25:19.020009 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75e0c21-20fa-4202-86a1-f122ec68102b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0cb44a17d205431793e9a8791e477a32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '559b3e713f5844b6ae37151d56dd419f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:25:19.020658 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75e0c21-20fa-4202-86a1-f122ec68102b service neutron] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:25:19.020923 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75e0c21-20fa-4202-86a1-f122ec68102b service neutron] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:25:19.021196 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75e0c21-20fa-4202-86a1-f122ec68102b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:25:19.030390 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f75e0c21-20fa-4202-86a1-f122ec68102b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:25:19.031836 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f75e0c21-20fa-4202-86a1-f122ec68102b service neutron] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:25:19.041343 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 727/1452] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Jun 15 10:25:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:25:21.212228 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79a6209-88e1-46ff-95ad-f8077d2a8c17 None None] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:25:21.212754 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79a6209-88e1-46ff-95ad-f8077d2a8c17 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:25:21.212754 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79a6209-88e1-46ff-95ad-f8077d2a8c17 None None] PATH_INFO: `/` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:25:21.214130 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 726/1453] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jun 15 10:25:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:25:21.219823 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58d8fca-15bc-42eb-8c5d-7edda0611a9b None None] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:25:21.220229 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58d8fca-15bc-42eb-8c5d-7edda0611a9b None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:25:21.220377 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58d8fca-15bc-42eb-8c5d-7edda0611a9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:25:21.238010 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.auth.core [None req-d58d8fca-15bc-42eb-8c5d-7edda0611a9b None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:25:21.239236 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-d58d8fca-15bc-42eb-8c5d-7edda0611a9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:25:21.246802 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 728/1454] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jun 15 10:25:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 10:25:21.283926 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79a6209-88e1-46ff-95ad-f8077d2a8c17 None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:25:21.294918 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79a6209-88e1-46ff-95ad-f8077d2a8c17 None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:25:21.299058 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e7ca7b-4873-474e-b43e-d9a75055fe06 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0cb44a17d205431793e9a8791e477a32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '559b3e713f5844b6ae37151d56dd419f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:25:21.299717 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e7ca7b-4873-474e-b43e-d9a75055fe06 service neutron] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:25:21.300001 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e7ca7b-4873-474e-b43e-d9a75055fe06 service neutron] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:25:21.300277 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e7ca7b-4873-474e-b43e-d9a75055fe06 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:25:21.309096 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e7ca7b-4873-474e-b43e-d9a75055fe06 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:25:21.310543 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e7ca7b-4873-474e-b43e-d9a75055fe06 service neutron] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:25:21.319818 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 727/1455] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Jun 15 10:25:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:25:23.475573 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7dc898a-eda4-42a7-90ae-4fb9e8d44311 None None] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:25:23.476164 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7dc898a-eda4-42a7-90ae-4fb9e8d44311 None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:25:23.476164 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7dc898a-eda4-42a7-90ae-4fb9e8d44311 None None] PATH_INFO: `/` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:25:23.477497 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 729/1456] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jun 15 10:25:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:25:23.482646 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467bef89-2d11-443c-835d-479e7a0e5306 None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:25:23.483049 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467bef89-2d11-443c-835d-479e7a0e5306 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:25:23.483280 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467bef89-2d11-443c-835d-479e7a0e5306 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:25:23.500924 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-467bef89-2d11-443c-835d-479e7a0e5306 None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:25:23.502124 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-467bef89-2d11-443c-835d-479e7a0e5306 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:25:23.509821 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 728/1457] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jun 15 10:25:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:25:23.547327 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7dc898a-eda4-42a7-90ae-4fb9e8d44311 None None] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:25:23.557786 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7dc898a-eda4-42a7-90ae-4fb9e8d44311 None None] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:25:23.561772 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a941f30e-1c70-4fd5-a3ab-3bbc90d78f61 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0cb44a17d205431793e9a8791e477a32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '559b3e713f5844b6ae37151d56dd419f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:25:23.562405 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a941f30e-1c70-4fd5-a3ab-3bbc90d78f61 service neutron] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:25:23.562683 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a941f30e-1c70-4fd5-a3ab-3bbc90d78f61 service neutron] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:25:23.562957 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a941f30e-1c70-4fd5-a3ab-3bbc90d78f61 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:25:23.571877 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a941f30e-1c70-4fd5-a3ab-3bbc90d78f61 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:25:23.573299 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a941f30e-1c70-4fd5-a3ab-3bbc90d78f61 service neutron] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:25:23.583207 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 730/1458] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Jun 15 10:25:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:25:25.748280 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134cfd98-cf10-4e4a-b088-13892aec348c None None] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:25:25.748829 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134cfd98-cf10-4e4a-b088-13892aec348c None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:25:25.748829 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134cfd98-cf10-4e4a-b088-13892aec348c None None] PATH_INFO: `/` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:25:25.750202 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 729/1459] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jun 15 10:25:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:25:25.755384 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea46e71c-c6eb-4405-9fbc-3620eb30d0d9 None None] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:25:25.755774 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea46e71c-c6eb-4405-9fbc-3620eb30d0d9 None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:25:25.755981 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea46e71c-c6eb-4405-9fbc-3620eb30d0d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:25:25.773568 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.auth.core [None req-ea46e71c-c6eb-4405-9fbc-3620eb30d0d9 None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:25:25.774764 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-ea46e71c-c6eb-4405-9fbc-3620eb30d0d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:25:25.783222 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 731/1460] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jun 15 10:25:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:25:25.820132 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134cfd98-cf10-4e4a-b088-13892aec348c None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:25:25.830847 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134cfd98-cf10-4e4a-b088-13892aec348c None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:25:25.834339 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2327881d-ed73-4f42-b082-6b9b26565546 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0cb44a17d205431793e9a8791e477a32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '559b3e713f5844b6ae37151d56dd419f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:25:25.834977 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2327881d-ed73-4f42-b082-6b9b26565546 service neutron] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:25:25.835296 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2327881d-ed73-4f42-b082-6b9b26565546 service neutron] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:25:25.835573 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2327881d-ed73-4f42-b082-6b9b26565546 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:25:25.844803 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2327881d-ed73-4f42-b082-6b9b26565546 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:25:25.846215 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2327881d-ed73-4f42-b082-6b9b26565546 service neutron] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:25:25.855683 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 730/1461] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Jun 15 10:25:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:25:28.000034 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940b8a78-df5f-4292-852e-4c4dfe4ffb3c None None] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:25:28.000613 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940b8a78-df5f-4292-852e-4c4dfe4ffb3c None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:25:28.000613 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940b8a78-df5f-4292-852e-4c4dfe4ffb3c None None] PATH_INFO: `/` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:25:28.001967 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 732/1462] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jun 15 10:25:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:25:28.007370 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ca27b2-f81d-4113-b481-bfdb9abe8cdf None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:25:28.007780 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ca27b2-f81d-4113-b481-bfdb9abe8cdf None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:25:28.007955 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ca27b2-f81d-4113-b481-bfdb9abe8cdf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:25:28.014699 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=79 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:25:28.027222 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-e6ca27b2-f81d-4113-b481-bfdb9abe8cdf None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:25:28.028460 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-e6ca27b2-f81d-4113-b481-bfdb9abe8cdf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:25:28.036248 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 731/1463] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jun 15 10:25:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:25:28.073558 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940b8a78-df5f-4292-852e-4c4dfe4ffb3c None None] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:25:28.075269 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-940b8a78-df5f-4292-852e-4c4dfe4ffb3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:25:28.083585 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=73 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:25:28.089647 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940b8a78-df5f-4292-852e-4c4dfe4ffb3c None None] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:25:28.094412 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3307cd0e-9e35-451b-9670-96cc04a47d46 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0cb44a17d205431793e9a8791e477a32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '559b3e713f5844b6ae37151d56dd419f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:25:28.095070 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3307cd0e-9e35-451b-9670-96cc04a47d46 service neutron] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:25:28.095373 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3307cd0e-9e35-451b-9670-96cc04a47d46 service neutron] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:25:28.095655 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3307cd0e-9e35-451b-9670-96cc04a47d46 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:25:28.104597 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3307cd0e-9e35-451b-9670-96cc04a47d46 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:25:28.106027 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3307cd0e-9e35-451b-9670-96cc04a47d46 service neutron] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:25:28.115347 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 733/1464] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Jun 15 10:25:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:25:30.264658 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184d2e5f-3155-4ab8-94f1-c4487952e048 None None] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:25:30.265232 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184d2e5f-3155-4ab8-94f1-c4487952e048 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:25:30.265232 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184d2e5f-3155-4ab8-94f1-c4487952e048 None None] PATH_INFO: `/` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:25:30.266576 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 732/1465] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jun 15 10:25:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:25:30.271850 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa27059-fef9-4668-9b0a-5ee9e0f78b56 None None] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:25:30.272296 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa27059-fef9-4668-9b0a-5ee9e0f78b56 None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:25:30.272392 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa27059-fef9-4668-9b0a-5ee9e0f78b56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:25:30.289782 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.auth.core [None req-4fa27059-fef9-4668-9b0a-5ee9e0f78b56 None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:25:30.291277 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-4fa27059-fef9-4668-9b0a-5ee9e0f78b56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:25:30.299357 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 734/1466] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jun 15 10:25:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:25:30.336016 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184d2e5f-3155-4ab8-94f1-c4487952e048 None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:25:30.346454 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184d2e5f-3155-4ab8-94f1-c4487952e048 None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:25:30.349928 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992665bc-d781-45b6-9bcd-02b251c5f7d6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0cb44a17d205431793e9a8791e477a32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '559b3e713f5844b6ae37151d56dd419f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:25:30.350557 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992665bc-d781-45b6-9bcd-02b251c5f7d6 service neutron] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:25:30.350837 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992665bc-d781-45b6-9bcd-02b251c5f7d6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:25:30.351137 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992665bc-d781-45b6-9bcd-02b251c5f7d6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:25:30.359752 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992665bc-d781-45b6-9bcd-02b251c5f7d6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:25:30.361153 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992665bc-d781-45b6-9bcd-02b251c5f7d6 service neutron] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:25:30.370714 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 733/1467] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Jun 15 10:25:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:25:32.494368 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77311afa-3b99-49d8-bfab-021d1cb67079 None None] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:25:32.494931 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77311afa-3b99-49d8-bfab-021d1cb67079 None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:25:32.494931 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77311afa-3b99-49d8-bfab-021d1cb67079 None None] PATH_INFO: `/` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:25:32.496339 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 735/1468] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jun 15 10:25:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:25:32.501606 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b7d38d-4b68-43fe-acb6-1f6381ec81e6 None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:25:32.502010 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b7d38d-4b68-43fe-acb6-1f6381ec81e6 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:25:32.502167 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b7d38d-4b68-43fe-acb6-1f6381ec81e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:25:32.519443 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-56b7d38d-4b68-43fe-acb6-1f6381ec81e6 None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:25:32.520639 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-56b7d38d-4b68-43fe-acb6-1f6381ec81e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:25:32.528293 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 734/1469] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jun 15 10:25:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:25:32.565334 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77311afa-3b99-49d8-bfab-021d1cb67079 None None] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:25:32.576233 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77311afa-3b99-49d8-bfab-021d1cb67079 None None] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:25:32.579835 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5b47ea-11dd-43e0-a8e1-3cd4ab30bd86 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0cb44a17d205431793e9a8791e477a32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '559b3e713f5844b6ae37151d56dd419f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:25:32.580501 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5b47ea-11dd-43e0-a8e1-3cd4ab30bd86 service neutron] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:25:32.580787 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5b47ea-11dd-43e0-a8e1-3cd4ab30bd86 service neutron] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:25:32.581066 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5b47ea-11dd-43e0-a8e1-3cd4ab30bd86 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:25:32.589923 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf5b47ea-11dd-43e0-a8e1-3cd4ab30bd86 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:25:32.591890 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf5b47ea-11dd-43e0-a8e1-3cd4ab30bd86 service neutron] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:25:32.601216 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 736/1470] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Jun 15 10:25:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 10:25:42.514420 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=11 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:25:42.596998 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=14 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:25:53.457189 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee748e7e-a901-4648-8945-e51f4cb95b07 None None] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:25:53.457761 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee748e7e-a901-4648-8945-e51f4cb95b07 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:25:53.457761 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee748e7e-a901-4648-8945-e51f4cb95b07 None None] PATH_INFO: `/` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:25:53.459135 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 735/1471] 10.222.0.49 () {54 vars in 926 bytes} [Wed Jun 15 10:25:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:25:53.464201 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5801c74c-260f-4eba-b363-8e353578bbdd None None] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:25:53.464614 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5801c74c-260f-4eba-b363-8e353578bbdd None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:25:53.464740 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5801c74c-260f-4eba-b363-8e353578bbdd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:25:53.483137 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.auth.core [None req-5801c74c-260f-4eba-b363-8e353578bbdd None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:25:53.484346 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-5801c74c-260f-4eba-b363-8e353578bbdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:25:53.492020 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 737/1472] 10.222.0.49 () {58 vars in 1023 bytes} [Wed Jun 15 10:25:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:25:53.538096 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39461991-b6ec-4247-b2d0-3db4ebd6f764 None None] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:25:53.538386 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39461991-b6ec-4247-b2d0-3db4ebd6f764 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:25:53.538662 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39461991-b6ec-4247-b2d0-3db4ebd6f764 None None] PATH_INFO: `/` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:25:53.540035 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 736/1473] 10.222.0.49 () {54 vars in 962 bytes} [Wed Jun 15 10:25:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:25:53.545841 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f17686-fb53-427c-893c-490f62b2757c None None] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:25:53.546131 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f17686-fb53-427c-893c-490f62b2757c None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:25:53.546406 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f17686-fb53-427c-893c-490f62b2757c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:25:53.563818 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.auth.core [None req-64f17686-fb53-427c-893c-490f62b2757c None None] MFA Rules not processed for user `91f2fe3be62e49548eae208e57bf2a93`. Rule list: `[]` (Enabled: `True`). {{(pid=80119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:25:53.565023 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-64f17686-fb53-427c-893c-490f62b2757c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:25:53.571382 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 738/1474] 10.222.0.49 () {58 vars in 1059 bytes} [Wed Jun 15 10:25:53 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 27 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:25:53.578605 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39461991-b6ec-4247-b2d0-3db4ebd6f764 None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:25:53.589038 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39461991-b6ec-4247-b2d0-3db4ebd6f764 None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:25:53.593018 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb49167-c19a-47d8-9930-7c0c404739ad service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91f2fe3be62e49548eae208e57bf2a93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '559b3e713f5844b6ae37151d56dd419f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:25:53.593638 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb49167-c19a-47d8-9930-7c0c404739ad service glance] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:25:53.593914 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb49167-c19a-47d8-9930-7c0c404739ad service glance] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:25:53.594185 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb49167-c19a-47d8-9930-7c0c404739ad service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:25:53.602874 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb49167-c19a-47d8-9930-7c0c404739ad service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:25:53.604327 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb49167-c19a-47d8-9930-7c0c404739ad service glance] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:25:53.614182 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 737/1475] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Jun 15 10:25:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 10:25:55.826686 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e3314a-cf63-44e2-82a6-f6b50ed07f32 None None] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:25:55.827182 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e3314a-cf63-44e2-82a6-f6b50ed07f32 None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:25:55.827289 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e3314a-cf63-44e2-82a6-f6b50ed07f32 None None] PATH_INFO: `/` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:25:55.828705 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 739/1476] 10.222.0.49 () {54 vars in 926 bytes} [Wed Jun 15 10:25:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:25:55.833910 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff08d2e-00d4-4fbe-b488-717425cd3be6 None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:25:55.834339 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff08d2e-00d4-4fbe-b488-717425cd3be6 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:25:55.834498 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff08d2e-00d4-4fbe-b488-717425cd3be6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:25:55.852604 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-2ff08d2e-00d4-4fbe-b488-717425cd3be6 None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:25:55.853788 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-2ff08d2e-00d4-4fbe-b488-717425cd3be6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:25:55.861523 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 738/1477] 10.222.0.49 () {58 vars in 1023 bytes} [Wed Jun 15 10:25:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:25:55.884308 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e3314a-cf63-44e2-82a6-f6b50ed07f32 None None] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:25:55.897209 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e3314a-cf63-44e2-82a6-f6b50ed07f32 None None] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:25:55.901616 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3d97e3-6058-4767-913e-57bcdbc9092d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:25:55.902263 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3d97e3-6058-4767-913e-57bcdbc9092d admin admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:25:55.902551 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3d97e3-6058-4767-913e-57bcdbc9092d admin admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:25:55.902833 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3d97e3-6058-4767-913e-57bcdbc9092d admin admin] PATH_INFO: `/v3/projects` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:25:55.903658 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f3d97e3-6058-4767-913e-57bcdbc9092d admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:25:55.905957 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: WARNING py.warnings [None req-0f3d97e3-6058-4767-913e-57bcdbc9092d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 15 10:25:55.905957 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: warnings.warn(msg) Jun 15 10:25:55.905957 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: Jun 15 10:25:55.907104 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f3d97e3-6058-4767-913e-57bcdbc9092d admin admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:25:55.926205 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 740/1478] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Jun 15 10:25:55 2022] GET /identity/v3/projects => generated 2553 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 10:25:58.053862 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfb0b00-245e-4d7a-8e1b-d73e5d7a756d None None] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:25:58.054419 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfb0b00-245e-4d7a-8e1b-d73e5d7a756d None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:25:58.054419 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfb0b00-245e-4d7a-8e1b-d73e5d7a756d None None] PATH_INFO: `/` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:25:58.055857 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 739/1479] 10.222.0.49 () {54 vars in 926 bytes} [Wed Jun 15 10:25:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:25:58.061575 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11071f0e-a441-43ce-a664-81334e1769b5 None None] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:25:58.062035 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11071f0e-a441-43ce-a664-81334e1769b5 None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:25:58.062146 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11071f0e-a441-43ce-a664-81334e1769b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:25:58.079767 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.auth.core [None req-11071f0e-a441-43ce-a664-81334e1769b5 None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:25:58.080978 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-11071f0e-a441-43ce-a664-81334e1769b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:25:58.088775 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 741/1480] 10.222.0.49 () {58 vars in 1023 bytes} [Wed Jun 15 10:25:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 10:25:58.131953 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dfb0b00-245e-4d7a-8e1b-d73e5d7a756d None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:25:58.146238 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dfb0b00-245e-4d7a-8e1b-d73e5d7a756d None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:25:58.151650 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9155bac-8b3a-4750-94c6-e74fe0f5f420 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ac54528370d4747b6e8777fba9d0c82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '559b3e713f5844b6ae37151d56dd419f', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:25:58.152292 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9155bac-8b3a-4750-94c6-e74fe0f5f420 service nova] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:25:58.152736 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9155bac-8b3a-4750-94c6-e74fe0f5f420 service nova] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:25:58.152863 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9155bac-8b3a-4750-94c6-e74fe0f5f420 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:25:58.160916 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9155bac-8b3a-4750-94c6-e74fe0f5f420 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:25:58.162532 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9155bac-8b3a-4750-94c6-e74fe0f5f420 service nova] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:25:58.172707 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 740/1481] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Jun 15 10:25:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:26:00.448923 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d28ab0-bd52-44d4-bd67-d0d371860b24 None None] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:26:00.449472 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d28ab0-bd52-44d4-bd67-d0d371860b24 None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:26:00.449472 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d28ab0-bd52-44d4-bd67-d0d371860b24 None None] PATH_INFO: `/` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:26:00.451365 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 742/1482] 10.222.0.49 () {54 vars in 926 bytes} [Wed Jun 15 10:26:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:26:00.456415 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329a7c99-4354-433f-a05d-150e5c3f8770 None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:26:00.456825 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329a7c99-4354-433f-a05d-150e5c3f8770 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:26:00.456976 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329a7c99-4354-433f-a05d-150e5c3f8770 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:26:00.474738 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-329a7c99-4354-433f-a05d-150e5c3f8770 None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:26:00.475965 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-329a7c99-4354-433f-a05d-150e5c3f8770 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:26:00.483769 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 741/1483] 10.222.0.49 () {58 vars in 1023 bytes} [Wed Jun 15 10:26:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:26:00.519850 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d28ab0-bd52-44d4-bd67-d0d371860b24 None None] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:26:00.530396 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d28ab0-bd52-44d4-bd67-d0d371860b24 None None] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:26:00.533843 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d179d16f-be8e-4bb0-9adf-ef2eb27249c4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0cb44a17d205431793e9a8791e477a32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '559b3e713f5844b6ae37151d56dd419f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:26:00.534471 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d179d16f-be8e-4bb0-9adf-ef2eb27249c4 service neutron] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:26:00.534754 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d179d16f-be8e-4bb0-9adf-ef2eb27249c4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:26:00.535029 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d179d16f-be8e-4bb0-9adf-ef2eb27249c4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:26:00.544126 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d179d16f-be8e-4bb0-9adf-ef2eb27249c4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:26:00.545547 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d179d16f-be8e-4bb0-9adf-ef2eb27249c4 service neutron] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:26:00.554665 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 743/1484] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Jun 15 10:26:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 10:26:02.727746 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13407ab-1f45-41a2-b005-5311082ab36a None None] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:26:02.728295 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13407ab-1f45-41a2-b005-5311082ab36a None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:26:02.728295 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13407ab-1f45-41a2-b005-5311082ab36a None None] PATH_INFO: `/` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:26:02.729692 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 742/1485] 10.222.0.49 () {54 vars in 926 bytes} [Wed Jun 15 10:26:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:26:02.735228 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d60fd0-d38a-46af-b755-939da844a3df None None] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:26:02.735657 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d60fd0-d38a-46af-b755-939da844a3df None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:26:02.735787 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d60fd0-d38a-46af-b755-939da844a3df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:26:02.753876 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.auth.core [None req-88d60fd0-d38a-46af-b755-939da844a3df None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:26:02.755073 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-88d60fd0-d38a-46af-b755-939da844a3df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:26:02.763581 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 744/1486] 10.222.0.49 () {58 vars in 1023 bytes} [Wed Jun 15 10:26:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:26:02.802482 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13407ab-1f45-41a2-b005-5311082ab36a None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:26:02.816287 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13407ab-1f45-41a2-b005-5311082ab36a None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:26:02.820117 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2b09cb-48ba-4829-8599-e0eeeae59330 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0cb44a17d205431793e9a8791e477a32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '559b3e713f5844b6ae37151d56dd419f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:26:02.820766 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2b09cb-48ba-4829-8599-e0eeeae59330 service neutron] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:26:02.821060 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2b09cb-48ba-4829-8599-e0eeeae59330 service neutron] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:26:02.821337 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2b09cb-48ba-4829-8599-e0eeeae59330 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:26:02.832602 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f2b09cb-48ba-4829-8599-e0eeeae59330 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:26:02.834061 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f2b09cb-48ba-4829-8599-e0eeeae59330 service neutron] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:26:02.843834 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 743/1487] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Jun 15 10:26:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:26:05.074989 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741b3e02-0f61-4fa7-8499-381cfa66d378 None None] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:26:05.076112 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741b3e02-0f61-4fa7-8499-381cfa66d378 None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:26:05.076112 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741b3e02-0f61-4fa7-8499-381cfa66d378 None None] PATH_INFO: `/` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:26:05.077507 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 745/1488] 10.222.0.49 () {54 vars in 926 bytes} [Wed Jun 15 10:26:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:26:05.082725 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacceb55-f297-4982-a032-8afbb3f731d7 None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:26:05.083177 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacceb55-f297-4982-a032-8afbb3f731d7 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:26:05.083320 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacceb55-f297-4982-a032-8afbb3f731d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:26:05.100679 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-bacceb55-f297-4982-a032-8afbb3f731d7 None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:26:05.101907 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-bacceb55-f297-4982-a032-8afbb3f731d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:26:05.109735 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 744/1489] 10.222.0.49 () {58 vars in 1023 bytes} [Wed Jun 15 10:26:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:26:05.149620 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741b3e02-0f61-4fa7-8499-381cfa66d378 None None] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:26:05.160145 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741b3e02-0f61-4fa7-8499-381cfa66d378 None None] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:26:05.163597 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc91b31f-cfd8-4be9-af8b-ae3c21dcdf38 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0cb44a17d205431793e9a8791e477a32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '559b3e713f5844b6ae37151d56dd419f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:26:05.164233 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc91b31f-cfd8-4be9-af8b-ae3c21dcdf38 service neutron] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:26:05.164514 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc91b31f-cfd8-4be9-af8b-ae3c21dcdf38 service neutron] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:26:05.164786 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc91b31f-cfd8-4be9-af8b-ae3c21dcdf38 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:26:05.173754 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc91b31f-cfd8-4be9-af8b-ae3c21dcdf38 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:26:05.175213 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc91b31f-cfd8-4be9-af8b-ae3c21dcdf38 service neutron] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:26:05.184881 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 746/1490] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Jun 15 10:26:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:26:07.486500 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6230238b-e21f-4783-9699-c47f6f8b2bee None None] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:26:07.487068 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6230238b-e21f-4783-9699-c47f6f8b2bee None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:26:07.487068 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6230238b-e21f-4783-9699-c47f6f8b2bee None None] PATH_INFO: `/` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:26:07.488464 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 745/1491] 10.222.0.49 () {54 vars in 926 bytes} [Wed Jun 15 10:26:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:26:07.493606 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff4be8a-7aef-4387-8c47-d72278e8d0e5 None None] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:26:07.494016 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff4be8a-7aef-4387-8c47-d72278e8d0e5 None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:26:07.494195 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff4be8a-7aef-4387-8c47-d72278e8d0e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:26:07.511860 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.auth.core [None req-dff4be8a-7aef-4387-8c47-d72278e8d0e5 None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:26:07.513617 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-dff4be8a-7aef-4387-8c47-d72278e8d0e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:26:07.521353 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 747/1492] 10.222.0.49 () {58 vars in 1023 bytes} [Wed Jun 15 10:26:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 10:26:07.559112 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6230238b-e21f-4783-9699-c47f6f8b2bee None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:26:07.571587 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6230238b-e21f-4783-9699-c47f6f8b2bee None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:26:07.575865 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd1d179-eb87-4da3-9b86-77011a1236f7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0cb44a17d205431793e9a8791e477a32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '559b3e713f5844b6ae37151d56dd419f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:26:07.576509 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd1d179-eb87-4da3-9b86-77011a1236f7 service neutron] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:26:07.576791 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd1d179-eb87-4da3-9b86-77011a1236f7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:26:07.577066 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd1d179-eb87-4da3-9b86-77011a1236f7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:26:07.586038 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd1d179-eb87-4da3-9b86-77011a1236f7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:26:07.587500 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd1d179-eb87-4da3-9b86-77011a1236f7 service neutron] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:26:07.596857 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 746/1493] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Jun 15 10:26:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:26:17.506888 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=33 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:26:17.592670 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=33 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:27:31.396238 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a71e8b-1408-48c9-a687-a81b65c62472 None None] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:27:31.396839 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a71e8b-1408-48c9-a687-a81b65c62472 None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:27:31.396839 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a71e8b-1408-48c9-a687-a81b65c62472 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:27:31.414668 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.auth.core [None req-99a71e8b-1408-48c9-a687-a81b65c62472 None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:27:31.416385 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-99a71e8b-1408-48c9-a687-a81b65c62472 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:27:31.424135 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 748/1494] 10.222.0.49 () {58 vars in 957 bytes} [Wed Jun 15 10:27:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:27:31.433965 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21dfbf3-db1e-4937-99c8-ff68b1453365 None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:27:31.434452 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21dfbf3-db1e-4937-99c8-ff68b1453365 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:27:31.434565 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21dfbf3-db1e-4937-99c8-ff68b1453365 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:27:31.452777 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-c21dfbf3-db1e-4937-99c8-ff68b1453365 None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:27:31.453972 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-c21dfbf3-db1e-4937-99c8-ff68b1453365 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:27:31.461713 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 747/1495] 10.222.0.49 () {58 vars in 957 bytes} [Wed Jun 15 10:27:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:27:31.466124 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a71e8b-1408-48c9-a687-a81b65c62472 None None] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:27:31.475740 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a71e8b-1408-48c9-a687-a81b65c62472 None None] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:27:31.480761 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b986f7-9c24-4f66-81c0-838f09ec8134 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:27:31.481416 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b986f7-9c24-4f66-81c0-838f09ec8134 admin admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:27:31.481703 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b986f7-9c24-4f66-81c0-838f09ec8134 admin admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:27:31.481981 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b986f7-9c24-4f66-81c0-838f09ec8134 admin admin] PATH_INFO: `/v3/domains` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:27:31.482696 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b986f7-9c24-4f66-81c0-838f09ec8134 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:27:31.483002 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b986f7-9c24-4f66-81c0-838f09ec8134 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:27:31.484055 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: WARNING py.warnings [None req-18b986f7-9c24-4f66-81c0-838f09ec8134 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 15 10:27:31.484055 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: warnings.warn(msg) Jun 15 10:27:31.484055 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: Jun 15 10:27:31.484930 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b986f7-9c24-4f66-81c0-838f09ec8134 admin admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:27:31.496791 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 749/1496] 10.222.0.49 () {58 vars in 1152 bytes} [Wed Jun 15 10:27:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 10:27:31.504642 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21dfbf3-db1e-4937-99c8-ff68b1453365 None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:27:31.513658 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21dfbf3-db1e-4937-99c8-ff68b1453365 None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:27:31.518163 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26916ce-9223-4c11-8944-95d1833a4085 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:27:31.518808 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26916ce-9223-4c11-8944-95d1833a4085 admin admin] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:27:31.519116 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26916ce-9223-4c11-8944-95d1833a4085 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:27:31.519391 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26916ce-9223-4c11-8944-95d1833a4085 admin admin] PATH_INFO: `/v3/projects` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:27:31.520077 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c26916ce-9223-4c11-8944-95d1833a4085 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:27:31.521829 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: WARNING py.warnings [None req-c26916ce-9223-4c11-8944-95d1833a4085 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 15 10:27:31.521829 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: warnings.warn(msg) Jun 15 10:27:31.521829 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: Jun 15 10:27:31.523017 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c26916ce-9223-4c11-8944-95d1833a4085 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:27:31.560092 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 748/1497] 10.222.0.49 () {60 vars in 1152 bytes} [Wed Jun 15 10:27:31 2022] POST /identity/v3/projects => generated 379 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 10:27:31.564374 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b986f7-9c24-4f66-81c0-838f09ec8134 admin admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:27:31.574777 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b986f7-9c24-4f66-81c0-838f09ec8134 admin admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:27:31.631864 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb31f58a-b576-4bf4-a0af-5c8f9196b1dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:27:31.632535 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb31f58a-b576-4bf4-a0af-5c8f9196b1dc admin admin] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:27:31.632826 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb31f58a-b576-4bf4-a0af-5c8f9196b1dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:27:31.633103 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb31f58a-b576-4bf4-a0af-5c8f9196b1dc admin admin] PATH_INFO: `/v3/users` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:27:31.633782 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb31f58a-b576-4bf4-a0af-5c8f9196b1dc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:27:31.634816 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: WARNING py.warnings [None req-cb31f58a-b576-4bf4-a0af-5c8f9196b1dc admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 15 10:27:31.634816 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: warnings.warn(msg) Jun 15 10:27:31.634816 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: Jun 15 10:27:31.636037 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb31f58a-b576-4bf4-a0af-5c8f9196b1dc admin admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:27:31.655845 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 750/1498] 10.222.0.49 () {60 vars in 1143 bytes} [Wed Jun 15 10:27:31 2022] POST /identity/v3/users => generated 290 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 10:27:31.659911 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26916ce-9223-4c11-8944-95d1833a4085 admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:27:31.671063 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26916ce-9223-4c11-8944-95d1833a4085 admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:27:31.675748 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7863b2-82a9-44e5-931e-6309e64be170 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:27:31.676402 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7863b2-82a9-44e5-931e-6309e64be170 admin admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:27:31.676743 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7863b2-82a9-44e5-931e-6309e64be170 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:27:31.676952 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7863b2-82a9-44e5-931e-6309e64be170 admin admin] PATH_INFO: `/v3/roles` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:27:31.677721 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f7863b2-82a9-44e5-931e-6309e64be170 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:27:31.679072 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: WARNING py.warnings [None req-1f7863b2-82a9-44e5-931e-6309e64be170 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 15 10:27:31.679072 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: warnings.warn(msg) Jun 15 10:27:31.679072 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: Jun 15 10:27:31.679963 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f7863b2-82a9-44e5-931e-6309e64be170 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:27:31.693567 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 749/1499] 10.222.0.49 () {58 vars in 1121 bytes} [Wed Jun 15 10:27:31 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 15 10:27:31.697351 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb31f58a-b576-4bf4-a0af-5c8f9196b1dc admin admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:27:31.707227 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb31f58a-b576-4bf4-a0af-5c8f9196b1dc admin admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:27:31.711884 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f91526-9e76-4ebd-84c4-14a33230d394 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:27:31.713028 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f91526-9e76-4ebd-84c4-14a33230d394 admin admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:27:31.713310 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f91526-9e76-4ebd-84c4-14a33230d394 admin admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:27:31.713582 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f91526-9e76-4ebd-84c4-14a33230d394 admin admin] PATH_INFO: `/v3/roles` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:27:31.714331 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f91526-9e76-4ebd-84c4-14a33230d394 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:27:31.715369 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: WARNING py.warnings [None req-e9f91526-9e76-4ebd-84c4-14a33230d394 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 15 10:27:31.715369 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: warnings.warn(msg) Jun 15 10:27:31.715369 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: Jun 15 10:27:31.716254 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f91526-9e76-4ebd-84c4-14a33230d394 admin admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:27:31.731080 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 751/1500] 10.222.0.49 () {58 vars in 1121 bytes} [Wed Jun 15 10:27:31 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 10:27:31.735818 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7863b2-82a9-44e5-931e-6309e64be170 admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:27:31.745544 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7863b2-82a9-44e5-931e-6309e64be170 admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:27:31.750093 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182bd3be-8120-43fc-9d32-07e7bbebc6bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:27:31.750971 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182bd3be-8120-43fc-9d32-07e7bbebc6bb admin admin] REQUEST_METHOD: `PUT` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:27:31.751287 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182bd3be-8120-43fc-9d32-07e7bbebc6bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:27:31.751561 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182bd3be-8120-43fc-9d32-07e7bbebc6bb admin admin] PATH_INFO: `/v3/projects/1bb46df93c6a4ab0affa5b2ea7602f01/users/075412bf6a494d0c899c54781d6cb2f0/roles/6876e57cc51c448097ab109d35b2a3da` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:27:31.765962 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-182bd3be-8120-43fc-9d32-07e7bbebc6bb admin admin] RBAC: Authorizing `identity:create_grant(project_id=1bb46df93c6a4ab0affa5b2ea7602f01, user_id=075412bf6a494d0c899c54781d6cb2f0, role_id=6876e57cc51c448097ab109d35b2a3da)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:27:31.766975 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: WARNING py.warnings [None req-182bd3be-8120-43fc-9d32-07e7bbebc6bb admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 15 10:27:31.766975 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: warnings.warn(msg) Jun 15 10:27:31.766975 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: Jun 15 10:27:31.768918 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-182bd3be-8120-43fc-9d32-07e7bbebc6bb admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:27:31.782451 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 750/1501] 10.222.0.49 () {60 vars in 1482 bytes} [Wed Jun 15 10:27:31 2022] PUT /identity/v3/projects/1bb46df93c6a4ab0affa5b2ea7602f01/users/075412bf6a494d0c899c54781d6cb2f0/roles/6876e57cc51c448097ab109d35b2a3da => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:27:31.794681 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83310c8e-5b35-465b-8db8-711d4e31ff76 None None] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:27:31.794864 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83310c8e-5b35-465b-8db8-711d4e31ff76 None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:27:31.795192 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83310c8e-5b35-465b-8db8-711d4e31ff76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:27:31.820377 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.auth.core [None req-83310c8e-5b35-465b-8db8-711d4e31ff76 None None] MFA Rules not processed for user `075412bf6a494d0c899c54781d6cb2f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:27:31.821574 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-83310c8e-5b35-465b-8db8-711d4e31ff76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:27:31.888306 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 752/1502] 10.222.0.49 () {58 vars in 957 bytes} [Wed Jun 15 10:27:31 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:27:31.901842 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e00e655-3711-413c-865b-13abe659176e None None] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:27:31.902125 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e00e655-3711-413c-865b-13abe659176e None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:27:31.902413 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e00e655-3711-413c-865b-13abe659176e None None] PATH_INFO: `/` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:27:31.903841 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 751/1503] 10.222.0.49 () {54 vars in 963 bytes} [Wed Jun 15 10:27:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:27:31.908783 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9cc14b-bd45-4e95-b696-4125a04993ce None None] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:27:31.909011 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9cc14b-bd45-4e95-b696-4125a04993ce None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:27:31.909298 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9cc14b-bd45-4e95-b696-4125a04993ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:27:31.927851 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.auth.core [None req-ea9cc14b-bd45-4e95-b696-4125a04993ce None None] MFA Rules not processed for user `20d1327553404b80aecee87c78fcb52b`. Rule list: `[]` (Enabled: `True`). {{(pid=80119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:27:31.929074 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-ea9cc14b-bd45-4e95-b696-4125a04993ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:27:31.971321 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:27:31.979459 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 753/1504] 10.222.0.49 () {58 vars in 1060 bytes} [Wed Jun 15 10:27:31 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 72 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:27:31.987077 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e00e655-3711-413c-865b-13abe659176e None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:27:31.997692 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e00e655-3711-413c-865b-13abe659176e None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:27:32.001235 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1090fb6c-6dae-488f-ace4-9f28c604e0d8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20d1327553404b80aecee87c78fcb52b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '559b3e713f5844b6ae37151d56dd419f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:27:32.001867 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1090fb6c-6dae-488f-ace4-9f28c604e0d8 service cinder] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:27:32.002150 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1090fb6c-6dae-488f-ace4-9f28c604e0d8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:27:32.002425 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1090fb6c-6dae-488f-ace4-9f28c604e0d8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:27:32.011149 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1090fb6c-6dae-488f-ace4-9f28c604e0d8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:27:32.012556 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1090fb6c-6dae-488f-ace4-9f28c604e0d8 service cinder] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:27:32.021142 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 752/1505] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Jun 15 10:27:31 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 10:27:32.062281 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c064ca-04ed-4829-a211-728eaab8a4cb None None] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:27:32.062577 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c064ca-04ed-4829-a211-728eaab8a4cb None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:27:32.062857 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c064ca-04ed-4829-a211-728eaab8a4cb None None] PATH_INFO: `/` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:27:32.064157 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 754/1506] 10.222.0.49 () {52 vars in 829 bytes} [Wed Jun 15 10:27:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:27:32.067737 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1090fb6c-6dae-488f-ace4-9f28c604e0d8 service cinder] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:27:32.128568 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1090fb6c-6dae-488f-ace4-9f28c604e0d8 service cinder] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:27:32.133117 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fcb0ce1-60ec-412c-909c-f356527ddbf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:27:32.133976 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fcb0ce1-60ec-412c-909c-f356527ddbf2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:27:32.134266 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fcb0ce1-60ec-412c-909c-f356527ddbf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:27:32.135029 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fcb0ce1-60ec-412c-909c-f356527ddbf2 admin admin] PATH_INFO: `/v3/users/075412bf6a494d0c899c54781d6cb2f0` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:27:32.136246 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fcb0ce1-60ec-412c-909c-f356527ddbf2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=075412bf6a494d0c899c54781d6cb2f0)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:27:32.137255 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: WARNING py.warnings [None req-3fcb0ce1-60ec-412c-909c-f356527ddbf2 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 15 10:27:32.137255 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: warnings.warn(msg) Jun 15 10:27:32.137255 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: Jun 15 10:27:32.138455 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fcb0ce1-60ec-412c-909c-f356527ddbf2 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:27:32.202619 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-3fcb0ce1-60ec-412c-909c-f356527ddbf2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '075412bf6a494d0c899c54781d6cb2f0'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:27:32.215394 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-3fcb0ce1-60ec-412c-909c-f356527ddbf2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '075412bf6a494d0c899c54781d6cb2f0'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:27:32.216164 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-3fcb0ce1-60ec-412c-909c-f356527ddbf2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '075412bf6a494d0c899c54781d6cb2f0'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:27:32.216794 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-3fcb0ce1-60ec-412c-909c-f356527ddbf2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '075412bf6a494d0c899c54781d6cb2f0'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:27:32.226355 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-3fcb0ce1-60ec-412c-909c-f356527ddbf2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '075412bf6a494d0c899c54781d6cb2f0'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:27:32.242395 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=88,keystone:INSERT=3,keystone:DELETE=9 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:27:32.251990 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 753/1507] 10.222.0.49 () {58 vars in 1223 bytes} [Wed Jun 15 10:27:32 2022] DELETE /identity/v3/users/075412bf6a494d0c899c54781d6cb2f0 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:27:32.256516 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4d1813-8d86-4954-829c-6c1b4798dc22 None None] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:27:32.256810 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4d1813-8d86-4954-829c-6c1b4798dc22 None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:27:32.257092 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4d1813-8d86-4954-829c-6c1b4798dc22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:27:32.274476 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.auth.core [None req-2e4d1813-8d86-4954-829c-6c1b4798dc22 None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:27:32.276316 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-2e4d1813-8d86-4954-829c-6c1b4798dc22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:27:32.334072 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 755/1508] 10.222.0.49 () {58 vars in 957 bytes} [Wed Jun 15 10:27:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:27:32.344325 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fcb0ce1-60ec-412c-909c-f356527ddbf2 admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:27:32.345726 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-3fcb0ce1-60ec-412c-909c-f356527ddbf2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:27:32.403080 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fcb0ce1-60ec-412c-909c-f356527ddbf2 admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:27:32.406720 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8a6161-13f0-46c0-a0f0-d1293b2d8b33 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0cb44a17d205431793e9a8791e477a32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '559b3e713f5844b6ae37151d56dd419f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:27:32.407410 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8a6161-13f0-46c0-a0f0-d1293b2d8b33 service neutron] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:27:32.407699 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8a6161-13f0-46c0-a0f0-d1293b2d8b33 service neutron] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:27:32.407977 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8a6161-13f0-46c0-a0f0-d1293b2d8b33 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:27:32.417846 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8a6161-13f0-46c0-a0f0-d1293b2d8b33 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:27:32.419319 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8a6161-13f0-46c0-a0f0-d1293b2d8b33 service neutron] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:27:32.428552 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 754/1509] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Jun 15 10:27:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:27:32.755332 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4d1813-8d86-4954-829c-6c1b4798dc22 None None] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:27:32.756854 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-2e4d1813-8d86-4954-829c-6c1b4798dc22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:27:32.773520 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4d1813-8d86-4954-829c-6c1b4798dc22 None None] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:27:32.776262 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08019d4-0248-4a0c-85dd-8775ec9c7a54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:27:32.777151 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08019d4-0248-4a0c-85dd-8775ec9c7a54 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:27:32.777925 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08019d4-0248-4a0c-85dd-8775ec9c7a54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:27:32.777925 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08019d4-0248-4a0c-85dd-8775ec9c7a54 admin admin] PATH_INFO: `/v3/projects/1bb46df93c6a4ab0affa5b2ea7602f01` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:27:32.778936 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a08019d4-0248-4a0c-85dd-8775ec9c7a54 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1bb46df93c6a4ab0affa5b2ea7602f01)` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:27:32.779993 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: WARNING py.warnings [None req-a08019d4-0248-4a0c-85dd-8775ec9c7a54 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 15 10:27:32.779993 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: warnings.warn(msg) Jun 15 10:27:32.779993 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: Jun 15 10:27:32.781185 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a08019d4-0248-4a0c-85dd-8775ec9c7a54 admin admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:27:32.845519 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-a08019d4-0248-4a0c-85dd-8775ec9c7a54 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1bb46df93c6a4ab0affa5b2ea7602f01', 'request_id': 'req-a08019d4-0248-4a0c-85dd-8775ec9c7a54'} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:27:32.855456 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-a08019d4-0248-4a0c-85dd-8775ec9c7a54 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1bb46df93c6a4ab0affa5b2ea7602f01', 'request_id': 'req-a08019d4-0248-4a0c-85dd-8775ec9c7a54'} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:27:32.861622 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-a08019d4-0248-4a0c-85dd-8775ec9c7a54 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1bb46df93c6a4ab0affa5b2ea7602f01', 'request_id': 'req-a08019d4-0248-4a0c-85dd-8775ec9c7a54'} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:27:32.880473 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-a08019d4-0248-4a0c-85dd-8775ec9c7a54 admin admin] The token cache is being invalidate because project 1bb46df93c6a4ab0affa5b2ea7602f01 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80119) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 10:27:32.880766 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-a08019d4-0248-4a0c-85dd-8775ec9c7a54 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:27:32.881489 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-a08019d4-0248-4a0c-85dd-8775ec9c7a54 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:27:32.883033 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 756/1510] 10.222.0.49 () {58 vars in 1232 bytes} [Wed Jun 15 10:27:32 2022] DELETE /identity/v3/projects/1bb46df93c6a4ab0affa5b2ea7602f01 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:27:42.424411 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:DELETE=1,keystone:SELECT=28 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:27:42.877356 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=55,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:29:13.911790 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da871978-b775-4d97-87b6-5b5ac8385994 None None] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:29:13.912372 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da871978-b775-4d97-87b6-5b5ac8385994 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:29:13.912372 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da871978-b775-4d97-87b6-5b5ac8385994 None None] PATH_INFO: `/` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:29:13.913714 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 755/1511] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jun 15 10:29:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:29:13.919300 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b51770-d9bf-4432-90f8-02fab0c4adc5 None None] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:29:13.919764 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b51770-d9bf-4432-90f8-02fab0c4adc5 None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:29:13.919874 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b51770-d9bf-4432-90f8-02fab0c4adc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:29:13.937357 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.auth.core [None req-83b51770-d9bf-4432-90f8-02fab0c4adc5 None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:29:13.938661 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-83b51770-d9bf-4432-90f8-02fab0c4adc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:29:13.995740 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 757/1512] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jun 15 10:29:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 10:29:14.164057 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da871978-b775-4d97-87b6-5b5ac8385994 None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:29:14.165757 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-da871978-b775-4d97-87b6-5b5ac8385994 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:29:14.238898 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da871978-b775-4d97-87b6-5b5ac8385994 None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:29:14.244291 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47577bc-7d37-4396-800c-efd3a53c3969 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ac54528370d4747b6e8777fba9d0c82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '559b3e713f5844b6ae37151d56dd419f', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:29:14.244934 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47577bc-7d37-4396-800c-efd3a53c3969 service nova] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:29:14.245220 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47577bc-7d37-4396-800c-efd3a53c3969 service nova] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:29:14.245499 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47577bc-7d37-4396-800c-efd3a53c3969 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:29:14.254143 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c47577bc-7d37-4396-800c-efd3a53c3969 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:29:14.255709 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c47577bc-7d37-4396-800c-efd3a53c3969 service nova] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:29:14.265714 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 756/1513] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Jun 15 10:29:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:29:15.145454 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2908bd9d-2476-4ba1-8487-7dc01adabafe None None] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:29:15.146028 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2908bd9d-2476-4ba1-8487-7dc01adabafe None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:29:15.146028 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2908bd9d-2476-4ba1-8487-7dc01adabafe None None] PATH_INFO: `/` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:29:15.147440 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 758/1514] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jun 15 10:29:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:29:15.152754 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3266478-2a2c-4e6d-995c-e9f0685cd67c None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:29:15.153195 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3266478-2a2c-4e6d-995c-e9f0685cd67c None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:29:15.153378 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3266478-2a2c-4e6d-995c-e9f0685cd67c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:29:15.171142 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-b3266478-2a2c-4e6d-995c-e9f0685cd67c None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:29:15.172345 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-b3266478-2a2c-4e6d-995c-e9f0685cd67c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:29:15.180095 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 757/1515] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jun 15 10:29:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:29:15.332954 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2908bd9d-2476-4ba1-8487-7dc01adabafe None None] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:29:15.350296 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2908bd9d-2476-4ba1-8487-7dc01adabafe None None] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:29:15.355638 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91bbf9db-32e1-4f93-b890-88e432abb008 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ac54528370d4747b6e8777fba9d0c82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '559b3e713f5844b6ae37151d56dd419f', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:29:15.356276 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bbf9db-32e1-4f93-b890-88e432abb008 service nova] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:29:15.356560 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bbf9db-32e1-4f93-b890-88e432abb008 service nova] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:29:15.356838 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bbf9db-32e1-4f93-b890-88e432abb008 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:29:15.364994 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91bbf9db-32e1-4f93-b890-88e432abb008 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:29:15.366591 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91bbf9db-32e1-4f93-b890-88e432abb008 service nova] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:29:15.376386 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 759/1516] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Jun 15 10:29:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:29:15.534401 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffeb379-3807-45a2-90c0-bfa7a8039445 None None] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:29:15.534685 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffeb379-3807-45a2-90c0-bfa7a8039445 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:29:15.534963 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffeb379-3807-45a2-90c0-bfa7a8039445 None None] PATH_INFO: `/v3` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:29:15.536574 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 758/1517] 10.222.0.62 () {54 vars in 925 bytes} [Wed Jun 15 10:29:15 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:29:15.543789 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48209892-19ea-4889-9fa0-6e4139dc44d6 None None] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:29:15.544080 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48209892-19ea-4889-9fa0-6e4139dc44d6 None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:29:15.544360 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48209892-19ea-4889-9fa0-6e4139dc44d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:29:15.564564 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.auth.core [None req-48209892-19ea-4889-9fa0-6e4139dc44d6 None None] MFA Rules not processed for user `dd03612a525644518517069b3fa43522`. Rule list: `[]` (Enabled: `True`). {{(pid=80119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:29:15.565803 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-48209892-19ea-4889-9fa0-6e4139dc44d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:29:15.626082 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 760/1518] 10.222.0.62 () {58 vars in 1015 bytes} [Wed Jun 15 10:29:15 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 10:29:24.957066 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0135388-b59f-448d-95f6-ac471f152211 None None] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:29:24.957575 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0135388-b59f-448d-95f6-ac471f152211 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:29:24.957575 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0135388-b59f-448d-95f6-ac471f152211 None None] PATH_INFO: `/` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:29:24.958904 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 759/1519] 10.222.0.49 () {54 vars in 960 bytes} [Wed Jun 15 10:29:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:29:24.964267 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cda682-ab2b-41a3-a19d-c91b61737c74 None None] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:29:24.964678 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cda682-ab2b-41a3-a19d-c91b61737c74 None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:29:24.964833 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cda682-ab2b-41a3-a19d-c91b61737c74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:29:24.983001 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.auth.core [None req-16cda682-ab2b-41a3-a19d-c91b61737c74 None None] MFA Rules not processed for user `dd03612a525644518517069b3fa43522`. Rule list: `[]` (Enabled: `True`). {{(pid=80119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:29:24.984226 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-16cda682-ab2b-41a3-a19d-c91b61737c74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:29:24.992693 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 761/1520] 10.222.0.49 () {58 vars in 1057 bytes} [Wed Jun 15 10:29:24 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:29:25.000596 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0135388-b59f-448d-95f6-ac471f152211 None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:29:25.013734 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0135388-b59f-448d-95f6-ac471f152211 None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:29:25.018598 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f064b3-fd59-4931-97a8-669661a728a8 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd03612a525644518517069b3fa43522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '559b3e713f5844b6ae37151d56dd419f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:29:25.019270 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f064b3-fd59-4931-97a8-669661a728a8 service placement] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:29:25.019559 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f064b3-fd59-4931-97a8-669661a728a8 service placement] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:29:25.019841 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f064b3-fd59-4931-97a8-669661a728a8 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:29:25.027931 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f064b3-fd59-4931-97a8-669661a728a8 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:29:25.029511 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f064b3-fd59-4931-97a8-669661a728a8 service placement] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:29:25.039276 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 760/1521] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Jun 15 10:29:24 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:29:26.657644 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f90255-f073-4853-aef9-c5f0fc731d34 None None] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:29:26.658229 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f90255-f073-4853-aef9-c5f0fc731d34 None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:29:26.658229 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f90255-f073-4853-aef9-c5f0fc731d34 None None] PATH_INFO: `/` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:29:26.659613 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 762/1522] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jun 15 10:29:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:29:26.664535 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939d0e37-1e22-4ed0-a24a-aaf475def842 None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:29:26.664960 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939d0e37-1e22-4ed0-a24a-aaf475def842 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:29:26.665091 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939d0e37-1e22-4ed0-a24a-aaf475def842 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:29:26.685684 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-939d0e37-1e22-4ed0-a24a-aaf475def842 None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:29:26.686885 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-939d0e37-1e22-4ed0-a24a-aaf475def842 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:29:26.694668 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 761/1523] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jun 15 10:29:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:29:26.848820 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f90255-f073-4853-aef9-c5f0fc731d34 None None] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:29:26.863276 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f90255-f073-4853-aef9-c5f0fc731d34 None None] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:29:26.868635 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63322c1c-a930-4f8b-a332-8bf153a471c9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ac54528370d4747b6e8777fba9d0c82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '559b3e713f5844b6ae37151d56dd419f', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:29:26.869272 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63322c1c-a930-4f8b-a332-8bf153a471c9 service nova] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:29:26.869558 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63322c1c-a930-4f8b-a332-8bf153a471c9 service nova] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:29:26.869835 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63322c1c-a930-4f8b-a332-8bf153a471c9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:29:26.878324 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63322c1c-a930-4f8b-a332-8bf153a471c9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:29:26.879950 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63322c1c-a930-4f8b-a332-8bf153a471c9 service nova] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:29:26.889900 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 763/1524] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Jun 15 10:29:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:29:27.742580 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188b2778-41b3-41f6-adac-07e02d33f296 None None] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:29:27.743172 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188b2778-41b3-41f6-adac-07e02d33f296 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:29:27.743172 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188b2778-41b3-41f6-adac-07e02d33f296 None None] PATH_INFO: `/` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:29:27.744523 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 762/1525] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jun 15 10:29:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:29:27.749393 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83949bcf-d1f2-4280-accb-78906781bbcc None None] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:29:27.749788 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83949bcf-d1f2-4280-accb-78906781bbcc None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:29:27.749964 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83949bcf-d1f2-4280-accb-78906781bbcc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:29:27.768095 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.auth.core [None req-83949bcf-d1f2-4280-accb-78906781bbcc None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:29:27.769286 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-83949bcf-d1f2-4280-accb-78906781bbcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:29:27.778045 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 764/1526] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jun 15 10:29:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:29:27.928114 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188b2778-41b3-41f6-adac-07e02d33f296 None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:29:27.942369 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188b2778-41b3-41f6-adac-07e02d33f296 None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:29:27.947834 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b53fabc-2462-4b9a-9f17-1e55a93048e0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ac54528370d4747b6e8777fba9d0c82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '559b3e713f5844b6ae37151d56dd419f', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:29:27.949036 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b53fabc-2462-4b9a-9f17-1e55a93048e0 service nova] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:29:27.949318 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b53fabc-2462-4b9a-9f17-1e55a93048e0 service nova] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:29:27.949594 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b53fabc-2462-4b9a-9f17-1e55a93048e0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:29:27.957750 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b53fabc-2462-4b9a-9f17-1e55a93048e0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:29:27.959352 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b53fabc-2462-4b9a-9f17-1e55a93048e0 service nova] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:29:27.969305 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 763/1527] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Jun 15 10:29:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:29:32.016681 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f0b218-d126-4495-82fc-fbcf2238a9ac None None] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:29:32.017210 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f0b218-d126-4495-82fc-fbcf2238a9ac None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:29:32.017210 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f0b218-d126-4495-82fc-fbcf2238a9ac None None] PATH_INFO: `/` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:29:32.018498 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 765/1528] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jun 15 10:29:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:29:32.023589 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70661aef-e546-4aff-b732-26ac19585881 None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:29:32.024000 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70661aef-e546-4aff-b732-26ac19585881 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:29:32.024164 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70661aef-e546-4aff-b732-26ac19585881 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:29:32.041720 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-70661aef-e546-4aff-b732-26ac19585881 None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:29:32.042908 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-70661aef-e546-4aff-b732-26ac19585881 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:29:32.050637 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 764/1529] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jun 15 10:29:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 10:29:32.087998 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f0b218-d126-4495-82fc-fbcf2238a9ac None None] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:29:32.089768 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-04f0b218-d126-4495-82fc-fbcf2238a9ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:29:32.146079 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f0b218-d126-4495-82fc-fbcf2238a9ac None None] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:29:32.149618 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc33c24-c886-4a7d-ae8c-2ec0d898989f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0cb44a17d205431793e9a8791e477a32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '559b3e713f5844b6ae37151d56dd419f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:29:32.150286 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc33c24-c886-4a7d-ae8c-2ec0d898989f service neutron] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:29:32.150572 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc33c24-c886-4a7d-ae8c-2ec0d898989f service neutron] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:29:32.150853 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc33c24-c886-4a7d-ae8c-2ec0d898989f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:29:32.159735 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc33c24-c886-4a7d-ae8c-2ec0d898989f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:29:32.161435 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc33c24-c886-4a7d-ae8c-2ec0d898989f service neutron] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:29:32.166303 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=100 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:29:32.171481 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 766/1530] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Jun 15 10:29:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:29:34.259938 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedae9fd-d796-44cf-b7e4-79ed2597ab46 None None] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:29:34.260510 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedae9fd-d796-44cf-b7e4-79ed2597ab46 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:29:34.260510 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedae9fd-d796-44cf-b7e4-79ed2597ab46 None None] PATH_INFO: `/` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:29:34.261895 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 765/1531] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jun 15 10:29:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:29:34.266997 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc739802-6bf6-4195-a1f7-1143d4b8ffb0 None None] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:29:34.267457 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc739802-6bf6-4195-a1f7-1143d4b8ffb0 None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:29:34.267623 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc739802-6bf6-4195-a1f7-1143d4b8ffb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:29:34.285910 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.auth.core [None req-bc739802-6bf6-4195-a1f7-1143d4b8ffb0 None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:29:34.287110 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-bc739802-6bf6-4195-a1f7-1143d4b8ffb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:29:34.295478 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 767/1532] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jun 15 10:29:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:29:34.330760 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eedae9fd-d796-44cf-b7e4-79ed2597ab46 None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:29:34.341877 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eedae9fd-d796-44cf-b7e4-79ed2597ab46 None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:29:34.345395 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65916604-3327-4b14-a98b-bbc8f4de6605 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0cb44a17d205431793e9a8791e477a32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '559b3e713f5844b6ae37151d56dd419f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:29:34.346028 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65916604-3327-4b14-a98b-bbc8f4de6605 service neutron] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:29:34.346311 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65916604-3327-4b14-a98b-bbc8f4de6605 service neutron] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:29:34.346590 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65916604-3327-4b14-a98b-bbc8f4de6605 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:29:34.355966 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65916604-3327-4b14-a98b-bbc8f4de6605 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:29:34.357392 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65916604-3327-4b14-a98b-bbc8f4de6605 service neutron] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:29:34.366883 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 766/1533] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Jun 15 10:29:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:29:38.046704 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5b8da8-5915-4cfc-952b-c349b9b2a2ee None None] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:29:38.047261 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5b8da8-5915-4cfc-952b-c349b9b2a2ee None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:29:38.047358 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5b8da8-5915-4cfc-952b-c349b9b2a2ee None None] PATH_INFO: `/` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:29:38.048741 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 768/1534] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jun 15 10:29:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:29:38.053942 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76707b57-a412-4d92-8d40-5054d03aebe6 None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:29:38.054395 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76707b57-a412-4d92-8d40-5054d03aebe6 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:29:38.054492 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76707b57-a412-4d92-8d40-5054d03aebe6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:29:38.072456 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-76707b57-a412-4d92-8d40-5054d03aebe6 None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:29:38.074260 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-76707b57-a412-4d92-8d40-5054d03aebe6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:29:38.081953 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 767/1535] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jun 15 10:29:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:29:38.345565 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5b8da8-5915-4cfc-952b-c349b9b2a2ee None None] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:29:38.347281 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-3a5b8da8-5915-4cfc-952b-c349b9b2a2ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:29:38.403261 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5b8da8-5915-4cfc-952b-c349b9b2a2ee None None] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:29:38.406925 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e5658d-99c1-4169-8785-bfef42e269a9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20d1327553404b80aecee87c78fcb52b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '559b3e713f5844b6ae37151d56dd419f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:29:38.407624 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e5658d-99c1-4169-8785-bfef42e269a9 service cinder] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:29:38.407934 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e5658d-99c1-4169-8785-bfef42e269a9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:29:38.408230 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e5658d-99c1-4169-8785-bfef42e269a9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:29:38.417822 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e5658d-99c1-4169-8785-bfef42e269a9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:29:38.419273 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e5658d-99c1-4169-8785-bfef42e269a9 service cinder] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:29:38.428414 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 769/1536] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Jun 15 10:29:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:29:40.579526 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dae74a5-8a8e-43d4-9255-f6cf5c78ea40 None None] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:29:40.580210 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dae74a5-8a8e-43d4-9255-f6cf5c78ea40 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:29:40.580210 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dae74a5-8a8e-43d4-9255-f6cf5c78ea40 None None] PATH_INFO: `/` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:29:40.581558 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 768/1537] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jun 15 10:29:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:29:40.587565 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac896332-161f-494f-91a2-31df3d17b870 None None] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:29:40.588005 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac896332-161f-494f-91a2-31df3d17b870 None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:29:40.588131 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac896332-161f-494f-91a2-31df3d17b870 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:29:40.606479 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.auth.core [None req-ac896332-161f-494f-91a2-31df3d17b870 None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:29:40.607713 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-ac896332-161f-494f-91a2-31df3d17b870 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:29:40.615490 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 770/1538] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jun 15 10:29:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:29:40.884228 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dae74a5-8a8e-43d4-9255-f6cf5c78ea40 None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:29:40.885975 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-1dae74a5-8a8e-43d4-9255-f6cf5c78ea40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:29:40.898675 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dae74a5-8a8e-43d4-9255-f6cf5c78ea40 None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:29:40.902665 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a6facc-ab26-4b5a-9c5d-499e08246715 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20d1327553404b80aecee87c78fcb52b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '559b3e713f5844b6ae37151d56dd419f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:29:40.903322 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a6facc-ab26-4b5a-9c5d-499e08246715 service cinder] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:29:40.903606 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a6facc-ab26-4b5a-9c5d-499e08246715 service cinder] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:29:40.903877 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a6facc-ab26-4b5a-9c5d-499e08246715 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:29:40.912835 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a6facc-ab26-4b5a-9c5d-499e08246715 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:29:40.914234 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a6facc-ab26-4b5a-9c5d-499e08246715 service cinder] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:29:40.925262 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 769/1539] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Jun 15 10:29:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:29:50.600753 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=32 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:29:50.920965 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=70 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:31:10.939275 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc633b9c-358e-4183-8c9b-7dba1a2083a9 None None] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:31:10.939939 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc633b9c-358e-4183-8c9b-7dba1a2083a9 None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:31:10.939939 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc633b9c-358e-4183-8c9b-7dba1a2083a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:31:10.967729 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.auth.core [None req-fc633b9c-358e-4183-8c9b-7dba1a2083a9 None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:31:10.969178 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-fc633b9c-358e-4183-8c9b-7dba1a2083a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:31:10.983143 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 771/1540] 10.222.0.49 () {58 vars in 957 bytes} [Wed Jun 15 10:31:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:31:10.998255 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4427abc-7271-48eb-ab87-90da86d71270 None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:31:10.998255 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4427abc-7271-48eb-ab87-90da86d71270 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:31:10.998255 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4427abc-7271-48eb-ab87-90da86d71270 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:31:11.003824 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5505ba39-2cc8-4da6-9f07-0fd7569c6049 None None] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:31:11.004332 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5505ba39-2cc8-4da6-9f07-0fd7569c6049 None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:31:11.004636 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5505ba39-2cc8-4da6-9f07-0fd7569c6049 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:31:11.036335 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-a4427abc-7271-48eb-ab87-90da86d71270 None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:31:11.037777 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.auth.core [None req-5505ba39-2cc8-4da6-9f07-0fd7569c6049 None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:31:11.037919 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-a4427abc-7271-48eb-ab87-90da86d71270 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:31:11.039125 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-5505ba39-2cc8-4da6-9f07-0fd7569c6049 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:31:11.055989 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 772/1541] 10.222.0.49 () {58 vars in 957 bytes} [Wed Jun 15 10:31:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:31:11.056194 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 770/1542] 10.222.0.49 () {58 vars in 957 bytes} [Wed Jun 15 10:31:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 65 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:31:11.057665 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9d2587-df81-4fc8-91e0-49edd7ffe3eb None None] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:31:11.057965 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9d2587-df81-4fc8-91e0-49edd7ffe3eb None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:31:11.058249 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9d2587-df81-4fc8-91e0-49edd7ffe3eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:31:11.061758 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4427abc-7271-48eb-ab87-90da86d71270 None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:31:11.079862 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4427abc-7271-48eb-ab87-90da86d71270 None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:31:11.088467 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.auth.core [None req-ea9d2587-df81-4fc8-91e0-49edd7ffe3eb None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:31:11.089711 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-ea9d2587-df81-4fc8-91e0-49edd7ffe3eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:31:11.091315 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b1379e-abbf-4fb7-b3f3-3e6c6f5a0258 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:31:11.092001 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b1379e-abbf-4fb7-b3f3-3e6c6f5a0258 admin admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:31:11.092283 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b1379e-abbf-4fb7-b3f3-3e6c6f5a0258 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:31:11.092554 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b1379e-abbf-4fb7-b3f3-3e6c6f5a0258 admin admin] PATH_INFO: `/v3/domains` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:31:11.093318 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b1379e-abbf-4fb7-b3f3-3e6c6f5a0258 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:31:11.093599 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b1379e-abbf-4fb7-b3f3-3e6c6f5a0258 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:31:11.094590 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: WARNING py.warnings [None req-79b1379e-abbf-4fb7-b3f3-3e6c6f5a0258 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 15 10:31:11.094590 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: warnings.warn(msg) Jun 15 10:31:11.094590 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: Jun 15 10:31:11.095503 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b1379e-abbf-4fb7-b3f3-3e6c6f5a0258 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:31:11.106838 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 773/1543] 10.222.0.49 () {58 vars in 957 bytes} [Wed Jun 15 10:31:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:31:11.108530 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e8ecbe-0212-4fe4-8a14-24b43f7ef928 None None] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:31:11.108813 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e8ecbe-0212-4fe4-8a14-24b43f7ef928 None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:31:11.109103 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e8ecbe-0212-4fe4-8a14-24b43f7ef928 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:31:11.109319 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 771/1544] 10.222.0.49 () {58 vars in 1152 bytes} [Wed Jun 15 10:31:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:31:11.110926 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388ea8d1-d0da-480f-a06a-efee776e2e4f None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:31:11.115440 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388ea8d1-d0da-480f-a06a-efee776e2e4f None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:31:11.115755 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388ea8d1-d0da-480f-a06a-efee776e2e4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:31:11.142210 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.auth.core [None req-87e8ecbe-0212-4fe4-8a14-24b43f7ef928 None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:31:11.143504 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-87e8ecbe-0212-4fe4-8a14-24b43f7ef928 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:31:11.148439 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-388ea8d1-d0da-480f-a06a-efee776e2e4f None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:31:11.149611 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-388ea8d1-d0da-480f-a06a-efee776e2e4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:31:11.157261 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 774/1545] 10.222.0.49 () {58 vars in 957 bytes} [Wed Jun 15 10:31:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:31:11.158961 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85283cbc-94f7-4fd9-8175-d69b1431b459 None None] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:31:11.159279 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85283cbc-94f7-4fd9-8175-d69b1431b459 None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:31:11.159559 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85283cbc-94f7-4fd9-8175-d69b1431b459 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:31:11.169729 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 772/1546] 10.222.0.49 () {58 vars in 957 bytes} [Wed Jun 15 10:31:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:31:11.177456 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c19f24-52b3-4029-a746-0e3d5e44d16f None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:31:11.177760 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c19f24-52b3-4029-a746-0e3d5e44d16f None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:31:11.178031 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c19f24-52b3-4029-a746-0e3d5e44d16f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:31:11.192007 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.auth.core [None req-85283cbc-94f7-4fd9-8175-d69b1431b459 None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:31:11.193227 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-85283cbc-94f7-4fd9-8175-d69b1431b459 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:31:11.198494 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-09c19f24-52b3-4029-a746-0e3d5e44d16f None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:31:11.199800 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-09c19f24-52b3-4029-a746-0e3d5e44d16f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:31:11.206948 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 775/1547] 10.222.0.49 () {58 vars in 957 bytes} [Wed Jun 15 10:31:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:31:11.207775 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85283cbc-94f7-4fd9-8175-d69b1431b459 None None] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:31:11.213592 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 773/1548] 10.222.0.49 () {58 vars in 957 bytes} [Wed Jun 15 10:31:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:31:11.214843 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af11885-866a-4d98-9cba-ae0e83c2b6a9 None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:31:11.215161 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af11885-866a-4d98-9cba-ae0e83c2b6a9 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:31:11.215423 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af11885-866a-4d98-9cba-ae0e83c2b6a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:31:11.218421 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85283cbc-94f7-4fd9-8175-d69b1431b459 None None] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:31:11.223363 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee33c4ca-f50f-404e-8108-2e9699dbd201 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:31:11.224096 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee33c4ca-f50f-404e-8108-2e9699dbd201 admin admin] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:31:11.224364 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee33c4ca-f50f-404e-8108-2e9699dbd201 admin admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:31:11.224642 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee33c4ca-f50f-404e-8108-2e9699dbd201 admin admin] PATH_INFO: `/v3/projects` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:31:11.225329 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee33c4ca-f50f-404e-8108-2e9699dbd201 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:31:11.226370 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: WARNING py.warnings [None req-ee33c4ca-f50f-404e-8108-2e9699dbd201 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 15 10:31:11.226370 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: warnings.warn(msg) Jun 15 10:31:11.226370 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: Jun 15 10:31:11.227639 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee33c4ca-f50f-404e-8108-2e9699dbd201 admin admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:31:11.233623 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-1af11885-866a-4d98-9cba-ae0e83c2b6a9 None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:31:11.235695 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-1af11885-866a-4d98-9cba-ae0e83c2b6a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:31:11.244861 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 774/1549] 10.222.0.49 () {58 vars in 957 bytes} [Wed Jun 15 10:31:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:31:11.245769 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af11885-866a-4d98-9cba-ae0e83c2b6a9 None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:31:11.255469 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af11885-866a-4d98-9cba-ae0e83c2b6a9 None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:31:11.256663 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 776/1550] 10.222.0.49 () {60 vars in 1152 bytes} [Wed Jun 15 10:31:11 2022] POST /identity/v3/projects => generated 393 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 10:31:11.258145 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc857e72-b4d1-474a-a10e-910c5cab619a None None] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:31:11.258437 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc857e72-b4d1-474a-a10e-910c5cab619a None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:31:11.258713 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc857e72-b4d1-474a-a10e-910c5cab619a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:31:11.277623 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.auth.core [None req-fc857e72-b4d1-474a-a10e-910c5cab619a None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:31:11.278830 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-fc857e72-b4d1-474a-a10e-910c5cab619a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:31:11.314186 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21be560-7637-4ee6-a25e-f78eb34c3864 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:31:11.314771 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21be560-7637-4ee6-a25e-f78eb34c3864 admin admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:31:11.315052 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21be560-7637-4ee6-a25e-f78eb34c3864 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:31:11.315354 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21be560-7637-4ee6-a25e-f78eb34c3864 admin admin] PATH_INFO: `/v3/domains` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:31:11.316071 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e21be560-7637-4ee6-a25e-f78eb34c3864 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:31:11.316377 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e21be560-7637-4ee6-a25e-f78eb34c3864 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:31:11.317960 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e21be560-7637-4ee6-a25e-f78eb34c3864 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:31:11.329905 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 775/1551] 10.222.0.49 () {58 vars in 1152 bytes} [Wed Jun 15 10:31:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:31:11.331273 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62da2226-9b1f-400b-8215-f52fe7345669 None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:31:11.331563 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62da2226-9b1f-400b-8215-f52fe7345669 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:31:11.331844 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62da2226-9b1f-400b-8215-f52fe7345669 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:31:11.346135 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 777/1552] 10.222.0.49 () {58 vars in 957 bytes} [Wed Jun 15 10:31:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:31:11.347483 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17705ec7-4b34-4a45-9740-3d4a381a2a63 None None] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:31:11.347772 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17705ec7-4b34-4a45-9740-3d4a381a2a63 None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:31:11.348099 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17705ec7-4b34-4a45-9740-3d4a381a2a63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:31:11.349608 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-62da2226-9b1f-400b-8215-f52fe7345669 None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:31:11.350833 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-62da2226-9b1f-400b-8215-f52fe7345669 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:31:11.358887 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 776/1553] 10.222.0.49 () {58 vars in 957 bytes} [Wed Jun 15 10:31:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:31:11.359723 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62da2226-9b1f-400b-8215-f52fe7345669 None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:31:11.366859 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.auth.core [None req-17705ec7-4b34-4a45-9740-3d4a381a2a63 None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:31:11.368108 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-17705ec7-4b34-4a45-9740-3d4a381a2a63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:31:11.371662 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62da2226-9b1f-400b-8215-f52fe7345669 None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:31:11.375938 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 778/1554] 10.222.0.49 () {58 vars in 957 bytes} [Wed Jun 15 10:31:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:31:11.376684 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42aec19-4afe-4a33-8b35-1f6e2e85af1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:31:11.377152 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc640ce-31f1-4419-88ed-9811f62e7f43 None None] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:31:11.377440 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc640ce-31f1-4419-88ed-9811f62e7f43 None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:31:11.377627 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42aec19-4afe-4a33-8b35-1f6e2e85af1b admin admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:31:11.377766 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc640ce-31f1-4419-88ed-9811f62e7f43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:31:11.377851 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42aec19-4afe-4a33-8b35-1f6e2e85af1b admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:31:11.378139 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42aec19-4afe-4a33-8b35-1f6e2e85af1b admin admin] PATH_INFO: `/v3/domains` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:31:11.378924 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c42aec19-4afe-4a33-8b35-1f6e2e85af1b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:31:11.379214 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c42aec19-4afe-4a33-8b35-1f6e2e85af1b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:31:11.380823 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c42aec19-4afe-4a33-8b35-1f6e2e85af1b admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:31:11.393519 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 777/1555] 10.222.0.49 () {58 vars in 1152 bytes} [Wed Jun 15 10:31:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:31:11.394342 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42aec19-4afe-4a33-8b35-1f6e2e85af1b admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:31:11.395932 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.auth.core [None req-7dc640ce-31f1-4419-88ed-9811f62e7f43 None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:31:11.397197 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-7dc640ce-31f1-4419-88ed-9811f62e7f43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:31:11.404200 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42aec19-4afe-4a33-8b35-1f6e2e85af1b admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:31:11.405967 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 779/1556] 10.222.0.49 () {58 vars in 957 bytes} [Wed Jun 15 10:31:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:31:11.406637 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc640ce-31f1-4419-88ed-9811f62e7f43 None None] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:31:11.409716 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719dd342-cb6f-4f71-b96d-fd777fda2b12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:31:11.410373 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719dd342-cb6f-4f71-b96d-fd777fda2b12 admin admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:31:11.410662 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719dd342-cb6f-4f71-b96d-fd777fda2b12 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:31:11.410941 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719dd342-cb6f-4f71-b96d-fd777fda2b12 admin admin] PATH_INFO: `/v3/domains` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:31:11.411707 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719dd342-cb6f-4f71-b96d-fd777fda2b12 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:31:11.412013 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719dd342-cb6f-4f71-b96d-fd777fda2b12 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:31:11.413606 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719dd342-cb6f-4f71-b96d-fd777fda2b12 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:31:11.417397 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc640ce-31f1-4419-88ed-9811f62e7f43 None None] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:31:11.422080 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d2bf79-6cfe-4602-b0c8-c3ecfcf328c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:31:11.422744 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d2bf79-6cfe-4602-b0c8-c3ecfcf328c5 admin admin] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:31:11.423030 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d2bf79-6cfe-4602-b0c8-c3ecfcf328c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:31:11.423355 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d2bf79-6cfe-4602-b0c8-c3ecfcf328c5 admin admin] PATH_INFO: `/v3/users` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:31:11.424175 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d2bf79-6cfe-4602-b0c8-c3ecfcf328c5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:31:11.426374 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 778/1557] 10.222.0.49 () {58 vars in 1152 bytes} [Wed Jun 15 10:31:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:31:11.426506 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d2bf79-6cfe-4602-b0c8-c3ecfcf328c5 admin admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:31:11.427302 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719dd342-cb6f-4f71-b96d-fd777fda2b12 admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:31:11.437083 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719dd342-cb6f-4f71-b96d-fd777fda2b12 admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:31:11.442765 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3a5777-2cbd-4531-b5a3-579598dd586f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:31:11.443431 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3a5777-2cbd-4531-b5a3-579598dd586f admin admin] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:31:11.443823 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3a5777-2cbd-4531-b5a3-579598dd586f admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:31:11.444105 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3a5777-2cbd-4531-b5a3-579598dd586f admin admin] PATH_INFO: `/v3/projects` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:31:11.444783 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a3a5777-2cbd-4531-b5a3-579598dd586f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:31:11.446652 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a3a5777-2cbd-4531-b5a3-579598dd586f admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:31:11.448148 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 780/1558] 10.222.0.49 () {60 vars in 1143 bytes} [Wed Jun 15 10:31:11 2022] POST /identity/v3/users => generated 297 bytes in 42 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 10:31:11.448148 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3b3b0e-3a0f-4603-938f-6a20ccf8bc40 None None] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:31:11.448148 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3b3b0e-3a0f-4603-938f-6a20ccf8bc40 None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:31:11.448148 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3b3b0e-3a0f-4603-938f-6a20ccf8bc40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:31:11.468357 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.auth.core [None req-9d3b3b0e-3a0f-4603-938f-6a20ccf8bc40 None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:31:11.469557 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-9d3b3b0e-3a0f-4603-938f-6a20ccf8bc40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:31:11.476519 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 779/1559] 10.222.0.49 () {60 vars in 1152 bytes} [Wed Jun 15 10:31:11 2022] POST /identity/v3/projects => generated 379 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 10:31:11.476519 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de3afcb-fda3-40fb-ab2e-268a7b23ab9e None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:31:11.476519 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de3afcb-fda3-40fb-ab2e-268a7b23ab9e None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:31:11.476519 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de3afcb-fda3-40fb-ab2e-268a7b23ab9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:31:11.477716 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 781/1560] 10.222.0.49 () {58 vars in 957 bytes} [Wed Jun 15 10:31:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:31:11.479487 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a0c129-5a04-4efd-be3e-d5fc974bfa57 None None] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:31:11.479775 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a0c129-5a04-4efd-be3e-d5fc974bfa57 None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:31:11.480073 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a0c129-5a04-4efd-be3e-d5fc974bfa57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:31:11.494896 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-8de3afcb-fda3-40fb-ab2e-268a7b23ab9e None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:31:11.496216 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-8de3afcb-fda3-40fb-ab2e-268a7b23ab9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:31:11.497654 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.auth.core [None req-70a0c129-5a04-4efd-be3e-d5fc974bfa57 None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:31:11.498831 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-70a0c129-5a04-4efd-be3e-d5fc974bfa57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:31:11.547121 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:31:11.557215 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 780/1561] 10.222.0.49 () {58 vars in 957 bytes} [Wed Jun 15 10:31:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:31:11.558011 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de3afcb-fda3-40fb-ab2e-268a7b23ab9e None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:31:11.558840 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 782/1562] 10.222.0.49 () {58 vars in 957 bytes} [Wed Jun 15 10:31:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:31:11.559760 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a0c129-5a04-4efd-be3e-d5fc974bfa57 None None] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:31:11.570082 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de3afcb-fda3-40fb-ab2e-268a7b23ab9e None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:31:11.570586 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a0c129-5a04-4efd-be3e-d5fc974bfa57 None None] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:31:11.575230 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74246d07-c45e-41f9-b3be-83ae2520511f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:31:11.575711 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50a39cf-7e19-4936-89b8-dc0d40542231 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:31:11.575883 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74246d07-c45e-41f9-b3be-83ae2520511f admin admin] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:31:11.576174 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74246d07-c45e-41f9-b3be-83ae2520511f admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:31:11.576356 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50a39cf-7e19-4936-89b8-dc0d40542231 admin admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:31:11.576462 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74246d07-c45e-41f9-b3be-83ae2520511f admin admin] PATH_INFO: `/v3/projects` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:31:11.576645 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50a39cf-7e19-4936-89b8-dc0d40542231 admin admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:31:11.576929 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50a39cf-7e19-4936-89b8-dc0d40542231 admin admin] PATH_INFO: `/v3/domains` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:31:11.577129 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74246d07-c45e-41f9-b3be-83ae2520511f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:31:11.577800 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a50a39cf-7e19-4936-89b8-dc0d40542231 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:31:11.578116 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a50a39cf-7e19-4936-89b8-dc0d40542231 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:31:11.579055 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74246d07-c45e-41f9-b3be-83ae2520511f admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:31:11.579735 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a50a39cf-7e19-4936-89b8-dc0d40542231 admin admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:31:11.582199 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:31:11.593547 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 783/1563] 10.222.0.49 () {58 vars in 1152 bytes} [Wed Jun 15 10:31:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:31:11.594644 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50a39cf-7e19-4936-89b8-dc0d40542231 admin admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:31:11.605388 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50a39cf-7e19-4936-89b8-dc0d40542231 admin admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:31:11.610101 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12bf3907-1a28-4c63-b428-f5ee8c24d9a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:31:11.610756 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bf3907-1a28-4c63-b428-f5ee8c24d9a5 admin admin] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:31:11.611073 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bf3907-1a28-4c63-b428-f5ee8c24d9a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:31:11.611450 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bf3907-1a28-4c63-b428-f5ee8c24d9a5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:31:11.615169 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 781/1564] 10.222.0.49 () {60 vars in 1152 bytes} [Wed Jun 15 10:31:11 2022] POST /identity/v3/projects => generated 383 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 10:31:11.615300 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12bf3907-1a28-4c63-b428-f5ee8c24d9a5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:31:11.615409 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74246d07-c45e-41f9-b3be-83ae2520511f admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:31:11.615499 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12bf3907-1a28-4c63-b428-f5ee8c24d9a5 admin admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:31:11.643685 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 784/1565] 10.222.0.49 () {60 vars in 1152 bytes} [Wed Jun 15 10:31:11 2022] POST /identity/v3/projects => generated 365 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 10:31:11.644638 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12bf3907-1a28-4c63-b428-f5ee8c24d9a5 admin admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:31:11.676763 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74246d07-c45e-41f9-b3be-83ae2520511f admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:31:11.707817 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12bf3907-1a28-4c63-b428-f5ee8c24d9a5 admin admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:31:11.712517 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b2c9e1-3323-4976-b213-d3ce0f626fc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:31:11.713715 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b2c9e1-3323-4976-b213-d3ce0f626fc4 admin admin] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:31:11.714002 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b2c9e1-3323-4976-b213-d3ce0f626fc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:31:11.714279 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b2c9e1-3323-4976-b213-d3ce0f626fc4 admin admin] PATH_INFO: `/v3/users` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:31:11.714965 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b2c9e1-3323-4976-b213-d3ce0f626fc4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:31:11.716938 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b2c9e1-3323-4976-b213-d3ce0f626fc4 admin admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:31:11.734563 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f18fc1-2ed0-46b6-bfe1-e0d91cc10483 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:31:11.735277 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f18fc1-2ed0-46b6-bfe1-e0d91cc10483 admin admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:31:11.735575 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f18fc1-2ed0-46b6-bfe1-e0d91cc10483 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:31:11.735985 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f18fc1-2ed0-46b6-bfe1-e0d91cc10483 admin admin] PATH_INFO: `/v3/roles` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:31:11.736590 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f18fc1-2ed0-46b6-bfe1-e0d91cc10483 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:31:11.738196 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f18fc1-2ed0-46b6-bfe1-e0d91cc10483 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:31:11.738354 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 785/1566] 10.222.0.49 () {60 vars in 1143 bytes} [Wed Jun 15 10:31:11 2022] POST /identity/v3/users => generated 290 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 10:31:11.739214 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b2c9e1-3323-4976-b213-d3ce0f626fc4 admin admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:31:11.751579 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 782/1567] 10.222.0.49 () {58 vars in 1121 bytes} [Wed Jun 15 10:31:11 2022] GET /identity/v3/roles => generated 1420 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 10:31:11.751579 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f18fc1-2ed0-46b6-bfe1-e0d91cc10483 admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:31:11.751870 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b2c9e1-3323-4976-b213-d3ce0f626fc4 admin admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:31:11.756058 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44288aa-72e6-4775-ad69-33cc4c879d2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:31:11.756706 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44288aa-72e6-4775-ad69-33cc4c879d2b admin admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:31:11.756988 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44288aa-72e6-4775-ad69-33cc4c879d2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:31:11.757265 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44288aa-72e6-4775-ad69-33cc4c879d2b admin admin] PATH_INFO: `/v3/domains` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:31:11.757975 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44288aa-72e6-4775-ad69-33cc4c879d2b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:31:11.758278 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44288aa-72e6-4775-ad69-33cc4c879d2b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:31:11.758772 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f18fc1-2ed0-46b6-bfe1-e0d91cc10483 admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:31:11.760006 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44288aa-72e6-4775-ad69-33cc4c879d2b admin admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:31:11.763494 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f31ad5-3a19-4737-a284-71f7282996b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:31:11.764132 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f31ad5-3a19-4737-a284-71f7282996b3 admin admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:31:11.764423 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f31ad5-3a19-4737-a284-71f7282996b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:31:11.764701 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f31ad5-3a19-4737-a284-71f7282996b3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:31:11.765417 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f31ad5-3a19-4737-a284-71f7282996b3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:31:11.765727 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f31ad5-3a19-4737-a284-71f7282996b3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:31:11.767331 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f31ad5-3a19-4737-a284-71f7282996b3 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:31:11.779971 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 786/1568] 10.222.0.49 () {58 vars in 1152 bytes} [Wed Jun 15 10:31:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:31:11.779971 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44288aa-72e6-4775-ad69-33cc4c879d2b admin admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:31:11.791933 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 783/1569] 10.222.0.49 () {58 vars in 1152 bytes} [Wed Jun 15 10:31:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:31:11.791933 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f31ad5-3a19-4737-a284-71f7282996b3 admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:31:11.792226 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44288aa-72e6-4775-ad69-33cc4c879d2b admin admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:31:11.796956 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5c372b-70a0-4821-8b7e-3570e376cb9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:31:11.797515 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f31ad5-3a19-4737-a284-71f7282996b3 admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:31:11.797639 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5c372b-70a0-4821-8b7e-3570e376cb9f admin admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:31:11.797895 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5c372b-70a0-4821-8b7e-3570e376cb9f admin admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:31:11.798174 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5c372b-70a0-4821-8b7e-3570e376cb9f admin admin] PATH_INFO: `/v3/domains` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:31:11.798951 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d5c372b-70a0-4821-8b7e-3570e376cb9f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:31:11.799306 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d5c372b-70a0-4821-8b7e-3570e376cb9f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:31:11.800852 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d5c372b-70a0-4821-8b7e-3570e376cb9f admin admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:31:11.802210 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa0f656-27d6-4da6-8a0d-bbc3991e6de4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:31:11.802893 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa0f656-27d6-4da6-8a0d-bbc3991e6de4 admin admin] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:31:11.803217 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa0f656-27d6-4da6-8a0d-bbc3991e6de4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:31:11.803502 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa0f656-27d6-4da6-8a0d-bbc3991e6de4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:31:11.804186 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa0f656-27d6-4da6-8a0d-bbc3991e6de4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:31:11.806067 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa0f656-27d6-4da6-8a0d-bbc3991e6de4 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:31:11.812721 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 787/1570] 10.222.0.49 () {58 vars in 1152 bytes} [Wed Jun 15 10:31:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:31:11.814065 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5c372b-70a0-4821-8b7e-3570e376cb9f admin admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:31:11.828875 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5c372b-70a0-4821-8b7e-3570e376cb9f admin admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:31:11.834199 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671683f2-84fe-4f72-a100-340ed4a398d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:31:11.834862 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671683f2-84fe-4f72-a100-340ed4a398d3 admin admin] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:31:11.835177 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671683f2-84fe-4f72-a100-340ed4a398d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:31:11.835474 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671683f2-84fe-4f72-a100-340ed4a398d3 admin admin] PATH_INFO: `/v3/users` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:31:11.836152 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-671683f2-84fe-4f72-a100-340ed4a398d3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:31:11.838760 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-671683f2-84fe-4f72-a100-340ed4a398d3 admin admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:31:11.840073 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 784/1571] 10.222.0.49 () {60 vars in 1152 bytes} [Wed Jun 15 10:31:11 2022] POST /identity/v3/projects => generated 385 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 10:31:11.840700 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa0f656-27d6-4da6-8a0d-bbc3991e6de4 admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:31:11.859568 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 788/1572] 10.222.0.49 () {60 vars in 1143 bytes} [Wed Jun 15 10:31:11 2022] POST /identity/v3/users => generated 292 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 10:31:11.860574 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671683f2-84fe-4f72-a100-340ed4a398d3 admin admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:31:11.870869 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671683f2-84fe-4f72-a100-340ed4a398d3 admin admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:31:11.895973 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:31:11.911944 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa0f656-27d6-4da6-8a0d-bbc3991e6de4 admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:31:11.916967 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4317ca94-555c-44e9-b274-9b9bb1a358c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:31:11.917656 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4317ca94-555c-44e9-b274-9b9bb1a358c0 admin admin] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:31:11.917940 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4317ca94-555c-44e9-b274-9b9bb1a358c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:31:11.918214 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4317ca94-555c-44e9-b274-9b9bb1a358c0 admin admin] PATH_INFO: `/v3/users` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:31:11.918906 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4317ca94-555c-44e9-b274-9b9bb1a358c0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:31:11.919951 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: WARNING py.warnings [None req-4317ca94-555c-44e9-b274-9b9bb1a358c0 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 15 10:31:11.919951 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: warnings.warn(msg) Jun 15 10:31:11.919951 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: Jun 15 10:31:11.921154 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4317ca94-555c-44e9-b274-9b9bb1a358c0 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:31:11.930475 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b64c939-c55c-42e2-bfc9-1acbf26485c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:31:11.931249 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b64c939-c55c-42e2-bfc9-1acbf26485c7 admin admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:31:11.932121 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b64c939-c55c-42e2-bfc9-1acbf26485c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:31:11.932395 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b64c939-c55c-42e2-bfc9-1acbf26485c7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:31:11.933189 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b64c939-c55c-42e2-bfc9-1acbf26485c7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:31:11.934730 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b64c939-c55c-42e2-bfc9-1acbf26485c7 admin admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:31:11.941036 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 785/1573] 10.222.0.49 () {60 vars in 1143 bytes} [Wed Jun 15 10:31:11 2022] POST /identity/v3/users => generated 283 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 10:31:11.941846 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4317ca94-555c-44e9-b274-9b9bb1a358c0 admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:31:11.948181 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 789/1574] 10.222.0.49 () {58 vars in 1121 bytes} [Wed Jun 15 10:31:11 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 10:31:11.949033 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b64c939-c55c-42e2-bfc9-1acbf26485c7 admin admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:31:11.952086 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4317ca94-555c-44e9-b274-9b9bb1a358c0 admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:31:11.956582 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3c37f4-74e0-400d-b6ff-2131b38d4e25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:31:11.957224 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3c37f4-74e0-400d-b6ff-2131b38d4e25 admin admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:31:11.958163 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3c37f4-74e0-400d-b6ff-2131b38d4e25 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:31:11.958403 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b64c939-c55c-42e2-bfc9-1acbf26485c7 admin admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:31:11.958504 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3c37f4-74e0-400d-b6ff-2131b38d4e25 admin admin] PATH_INFO: `/v3/roles` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:31:11.959185 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3c37f4-74e0-400d-b6ff-2131b38d4e25 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:31:11.960788 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3c37f4-74e0-400d-b6ff-2131b38d4e25 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:31:11.963153 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6689ba-200e-440f-8d39-c0992f7f8524 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:31:11.963793 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6689ba-200e-440f-8d39-c0992f7f8524 admin admin] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:31:11.964081 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6689ba-200e-440f-8d39-c0992f7f8524 admin admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:31:11.964360 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6689ba-200e-440f-8d39-c0992f7f8524 admin admin] PATH_INFO: `/v3/projects` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:31:11.965038 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f6689ba-200e-440f-8d39-c0992f7f8524 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:31:11.966927 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f6689ba-200e-440f-8d39-c0992f7f8524 admin admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:31:11.973576 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 786/1575] 10.222.0.49 () {58 vars in 1121 bytes} [Wed Jun 15 10:31:11 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 10:31:11.973576 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3c37f4-74e0-400d-b6ff-2131b38d4e25 admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:31:11.981896 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3c37f4-74e0-400d-b6ff-2131b38d4e25 admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:31:11.985147 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:31:11.987505 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a891d23-684a-4b52-aa3d-7434d999fa26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:31:11.988459 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a891d23-684a-4b52-aa3d-7434d999fa26 admin admin] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:31:11.988866 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a891d23-684a-4b52-aa3d-7434d999fa26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:31:11.989266 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a891d23-684a-4b52-aa3d-7434d999fa26 admin admin] PATH_INFO: `/v3/projects` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:31:11.990130 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a891d23-684a-4b52-aa3d-7434d999fa26 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:31:11.992858 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a891d23-684a-4b52-aa3d-7434d999fa26 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:31:12.002236 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 790/1576] 10.222.0.49 () {60 vars in 1152 bytes} [Wed Jun 15 10:31:11 2022] POST /identity/v3/projects => generated 365 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 10:31:12.002236 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6689ba-200e-440f-8d39-c0992f7f8524 admin admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:31:12.023170 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 787/1577] 10.222.0.49 () {60 vars in 1152 bytes} [Wed Jun 15 10:31:11 2022] POST /identity/v3/projects => generated 387 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 10:31:12.023170 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a891d23-684a-4b52-aa3d-7434d999fa26 admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:31:12.065150 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6689ba-200e-440f-8d39-c0992f7f8524 admin admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:31:12.081209 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a891d23-684a-4b52-aa3d-7434d999fa26 admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:31:12.085659 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3711f864-bc31-481f-bd1c-218dd8d16321 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:31:12.086321 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3711f864-bc31-481f-bd1c-218dd8d16321 admin admin] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:31:12.086604 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3711f864-bc31-481f-bd1c-218dd8d16321 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:31:12.086876 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3711f864-bc31-481f-bd1c-218dd8d16321 admin admin] PATH_INFO: `/v3/users` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:31:12.087586 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3711f864-bc31-481f-bd1c-218dd8d16321 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:31:12.089528 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3711f864-bc31-481f-bd1c-218dd8d16321 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:31:12.110603 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 788/1578] 10.222.0.49 () {60 vars in 1143 bytes} [Wed Jun 15 10:31:12 2022] POST /identity/v3/users => generated 293 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 10:31:12.111431 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3711f864-bc31-481f-bd1c-218dd8d16321 admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:31:12.121081 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3711f864-bc31-481f-bd1c-218dd8d16321 admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:31:12.122780 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234a5315-2d81-48d4-ad9c-b5e093a5fb60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:31:12.123474 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234a5315-2d81-48d4-ad9c-b5e093a5fb60 admin admin] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:31:12.123766 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234a5315-2d81-48d4-ad9c-b5e093a5fb60 admin admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:31:12.124062 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234a5315-2d81-48d4-ad9c-b5e093a5fb60 admin admin] PATH_INFO: `/v3/projects` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:31:12.124719 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-234a5315-2d81-48d4-ad9c-b5e093a5fb60 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:31:12.125603 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40cbfb88-8812-4db5-8c2a-9d28a9383f46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:31:12.126231 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40cbfb88-8812-4db5-8c2a-9d28a9383f46 admin admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:31:12.126511 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40cbfb88-8812-4db5-8c2a-9d28a9383f46 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:31:12.126619 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-234a5315-2d81-48d4-ad9c-b5e093a5fb60 admin admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:31:12.126784 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40cbfb88-8812-4db5-8c2a-9d28a9383f46 admin admin] PATH_INFO: `/v3/roles` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:31:12.127534 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40cbfb88-8812-4db5-8c2a-9d28a9383f46 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:31:12.129085 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40cbfb88-8812-4db5-8c2a-9d28a9383f46 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:31:12.142197 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 789/1579] 10.222.0.49 () {58 vars in 1121 bytes} [Wed Jun 15 10:31:12 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 10:31:12.142197 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40cbfb88-8812-4db5-8c2a-9d28a9383f46 admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:31:12.150026 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40cbfb88-8812-4db5-8c2a-9d28a9383f46 admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:31:12.154371 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 791/1580] 10.222.0.49 () {60 vars in 1152 bytes} [Wed Jun 15 10:31:11 2022] POST /identity/v3/projects => generated 387 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 10:31:12.154534 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3ddfe4-71aa-402a-95fe-6dfba689c1c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:31:12.155200 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3ddfe4-71aa-402a-95fe-6dfba689c1c6 admin admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:31:12.155501 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3ddfe4-71aa-402a-95fe-6dfba689c1c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:31:12.155756 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3ddfe4-71aa-402a-95fe-6dfba689c1c6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:31:12.156458 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3ddfe4-71aa-402a-95fe-6dfba689c1c6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:31:12.158059 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3ddfe4-71aa-402a-95fe-6dfba689c1c6 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:31:12.158396 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234a5315-2d81-48d4-ad9c-b5e093a5fb60 admin admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:31:12.169462 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 790/1581] 10.222.0.49 () {58 vars in 1121 bytes} [Wed Jun 15 10:31:12 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 10:31:12.170385 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3ddfe4-71aa-402a-95fe-6dfba689c1c6 admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:31:12.180290 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3ddfe4-71aa-402a-95fe-6dfba689c1c6 admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:31:12.221045 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234a5315-2d81-48d4-ad9c-b5e093a5fb60 admin admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:31:12.225649 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3221d9fb-f083-4b80-abbb-831518e51887 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:31:12.226315 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3221d9fb-f083-4b80-abbb-831518e51887 admin admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:31:12.226600 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3221d9fb-f083-4b80-abbb-831518e51887 admin admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:31:12.226879 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3221d9fb-f083-4b80-abbb-831518e51887 admin admin] PATH_INFO: `/v3/roles` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:31:12.227631 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3221d9fb-f083-4b80-abbb-831518e51887 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:31:12.229834 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3221d9fb-f083-4b80-abbb-831518e51887 admin admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:31:12.235952 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02d3b15-0869-480c-8c87-9f8513a7be2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:31:12.236846 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02d3b15-0869-480c-8c87-9f8513a7be2d admin admin] REQUEST_METHOD: `PUT` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:31:12.237127 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02d3b15-0869-480c-8c87-9f8513a7be2d admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:31:12.237403 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02d3b15-0869-480c-8c87-9f8513a7be2d admin admin] PATH_INFO: `/v3/projects/993c57bae00b4a0db1d8de0f5017630a/users/6d152f74395c4b6cb889058f612d502f/roles/6876e57cc51c448097ab109d35b2a3da` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:31:12.240001 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 792/1582] 10.222.0.49 () {58 vars in 1121 bytes} [Wed Jun 15 10:31:12 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 10:31:12.240777 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3221d9fb-f083-4b80-abbb-831518e51887 admin admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:31:12.250545 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3221d9fb-f083-4b80-abbb-831518e51887 admin admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:31:12.255401 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54216a4c-dde6-43f3-bee7-2f7dfc2c633a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:31:12.255603 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02d3b15-0869-480c-8c87-9f8513a7be2d admin admin] RBAC: Authorizing `identity:create_grant(project_id=993c57bae00b4a0db1d8de0f5017630a, user_id=6d152f74395c4b6cb889058f612d502f, role_id=6876e57cc51c448097ab109d35b2a3da)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:31:12.256038 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54216a4c-dde6-43f3-bee7-2f7dfc2c633a admin admin] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:31:12.256326 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54216a4c-dde6-43f3-bee7-2f7dfc2c633a admin admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:31:12.256608 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54216a4c-dde6-43f3-bee7-2f7dfc2c633a admin admin] PATH_INFO: `/v3/users` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:31:12.257277 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54216a4c-dde6-43f3-bee7-2f7dfc2c633a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:31:12.258245 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02d3b15-0869-480c-8c87-9f8513a7be2d admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:31:12.259191 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54216a4c-dde6-43f3-bee7-2f7dfc2c633a admin admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:31:12.260789 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:31:12.272630 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 791/1583] 10.222.0.49 () {60 vars in 1482 bytes} [Wed Jun 15 10:31:12 2022] PUT /identity/v3/projects/993c57bae00b4a0db1d8de0f5017630a/users/6d152f74395c4b6cb889058f612d502f/roles/6876e57cc51c448097ab109d35b2a3da => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:31:12.273516 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02d3b15-0869-480c-8c87-9f8513a7be2d admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:31:12.285970 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 793/1584] 10.222.0.49 () {60 vars in 1143 bytes} [Wed Jun 15 10:31:12 2022] POST /identity/v3/users => generated 283 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 10:31:12.286789 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54216a4c-dde6-43f3-bee7-2f7dfc2c633a admin admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:31:12.293460 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:INSERT=5,keystone:SELECT=95 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:31:12.297709 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54216a4c-dde6-43f3-bee7-2f7dfc2c633a admin admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:31:12.339728 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02d3b15-0869-480c-8c87-9f8513a7be2d admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:31:12.344288 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b03758-1d6f-4a1d-aae6-a451714d4dc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:31:12.344963 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b03758-1d6f-4a1d-aae6-a451714d4dc9 admin admin] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:31:12.345526 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b03758-1d6f-4a1d-aae6-a451714d4dc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:31:12.345526 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b03758-1d6f-4a1d-aae6-a451714d4dc9 admin admin] PATH_INFO: `/v3/users` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:31:12.346215 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b03758-1d6f-4a1d-aae6-a451714d4dc9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:31:12.348191 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b03758-1d6f-4a1d-aae6-a451714d4dc9 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:31:12.354552 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3cff52-55de-4705-85a4-3c3b3a9fa0e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:31:12.355387 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3cff52-55de-4705-85a4-3c3b3a9fa0e5 admin admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:31:12.355565 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3cff52-55de-4705-85a4-3c3b3a9fa0e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:31:12.355844 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3cff52-55de-4705-85a4-3c3b3a9fa0e5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:31:12.356568 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d3cff52-55de-4705-85a4-3c3b3a9fa0e5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:31:12.358159 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d3cff52-55de-4705-85a4-3c3b3a9fa0e5 admin admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:31:12.368537 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 794/1585] 10.222.0.49 () {58 vars in 1121 bytes} [Wed Jun 15 10:31:12 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 10:31:12.372176 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3cff52-55de-4705-85a4-3c3b3a9fa0e5 admin admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:31:12.372701 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 792/1586] 10.222.0.49 () {60 vars in 1143 bytes} [Wed Jun 15 10:31:12 2022] POST /identity/v3/users => generated 294 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 10:31:12.372701 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b03758-1d6f-4a1d-aae6-a451714d4dc9 admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:31:12.380909 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b03758-1d6f-4a1d-aae6-a451714d4dc9 admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:31:12.381637 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3cff52-55de-4705-85a4-3c3b3a9fa0e5 admin admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:31:12.385493 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1850bd08-cf86-4293-b0f1-0d292346329b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:31:12.386126 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669c88ae-34a1-4e81-ad5e-cd58af0d09a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:31:12.386243 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1850bd08-cf86-4293-b0f1-0d292346329b admin admin] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:31:12.386422 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1850bd08-cf86-4293-b0f1-0d292346329b admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:31:12.386699 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1850bd08-cf86-4293-b0f1-0d292346329b admin admin] PATH_INFO: `/v3/users` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:31:12.386810 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669c88ae-34a1-4e81-ad5e-cd58af0d09a3 admin admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:31:12.387189 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669c88ae-34a1-4e81-ad5e-cd58af0d09a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:31:12.387413 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1850bd08-cf86-4293-b0f1-0d292346329b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:31:12.387529 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669c88ae-34a1-4e81-ad5e-cd58af0d09a3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:31:12.388172 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-669c88ae-34a1-4e81-ad5e-cd58af0d09a3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:31:12.389316 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1850bd08-cf86-4293-b0f1-0d292346329b admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:31:12.389743 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-669c88ae-34a1-4e81-ad5e-cd58af0d09a3 admin admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:31:12.400852 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 795/1587] 10.222.0.49 () {58 vars in 1121 bytes} [Wed Jun 15 10:31:12 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 10:31:12.401873 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669c88ae-34a1-4e81-ad5e-cd58af0d09a3 admin admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:31:12.410739 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 793/1588] 10.222.0.49 () {60 vars in 1143 bytes} [Wed Jun 15 10:31:12 2022] POST /identity/v3/users => generated 294 bytes in 41 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 10:31:12.411768 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1850bd08-cf86-4293-b0f1-0d292346329b admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:31:12.411965 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669c88ae-34a1-4e81-ad5e-cd58af0d09a3 admin admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:31:12.417089 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8eaaae-f9c7-4181-bb6d-0e966af6b68b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:31:12.417744 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8eaaae-f9c7-4181-bb6d-0e966af6b68b admin admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:31:12.418032 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8eaaae-f9c7-4181-bb6d-0e966af6b68b admin admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:31:12.418314 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8eaaae-f9c7-4181-bb6d-0e966af6b68b admin admin] PATH_INFO: `/v3/roles` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:31:12.419034 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f8eaaae-f9c7-4181-bb6d-0e966af6b68b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:31:12.420657 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f8eaaae-f9c7-4181-bb6d-0e966af6b68b admin admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:31:12.422101 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1850bd08-cf86-4293-b0f1-0d292346329b admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:31:12.426896 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40746743-3728-439e-9e18-f05cb4e8836f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:31:12.427805 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40746743-3728-439e-9e18-f05cb4e8836f admin admin] REQUEST_METHOD: `PUT` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:31:12.428091 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40746743-3728-439e-9e18-f05cb4e8836f admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:31:12.428370 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40746743-3728-439e-9e18-f05cb4e8836f admin admin] PATH_INFO: `/v3/projects/b8df5f1986694e8f9ee6f55ef3995621/users/38e2794bd474425d9d75f21c1929972b/roles/6876e57cc51c448097ab109d35b2a3da` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:31:12.432757 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 796/1589] 10.222.0.49 () {58 vars in 1121 bytes} [Wed Jun 15 10:31:12 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 10:31:12.437036 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662fe92e-2157-484e-9537-73868ada5cb8 None None] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:31:12.437755 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662fe92e-2157-484e-9537-73868ada5cb8 None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:31:12.438047 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662fe92e-2157-484e-9537-73868ada5cb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:31:12.443734 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40746743-3728-439e-9e18-f05cb4e8836f admin admin] RBAC: Authorizing `identity:create_grant(project_id=b8df5f1986694e8f9ee6f55ef3995621, user_id=38e2794bd474425d9d75f21c1929972b, role_id=6876e57cc51c448097ab109d35b2a3da)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:31:12.446414 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40746743-3728-439e-9e18-f05cb4e8836f admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:31:12.454447 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.auth.core [None req-662fe92e-2157-484e-9537-73868ada5cb8 None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:31:12.455694 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-662fe92e-2157-484e-9537-73868ada5cb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:31:12.459508 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 794/1590] 10.222.0.49 () {60 vars in 1482 bytes} [Wed Jun 15 10:31:12 2022] PUT /identity/v3/projects/b8df5f1986694e8f9ee6f55ef3995621/users/38e2794bd474425d9d75f21c1929972b/roles/6876e57cc51c448097ab109d35b2a3da => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:31:12.464320 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40746743-3728-439e-9e18-f05cb4e8836f admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:31:12.515692 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 797/1591] 10.222.0.49 () {58 vars in 957 bytes} [Wed Jun 15 10:31:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:31:12.516567 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662fe92e-2157-484e-9537-73868ada5cb8 None None] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:31:12.524036 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40746743-3728-439e-9e18-f05cb4e8836f admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:31:12.526162 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662fe92e-2157-484e-9537-73868ada5cb8 None None] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:31:12.528718 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8122d67d-71c4-45a8-a66a-e777ee8afa3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:31:12.529390 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8122d67d-71c4-45a8-a66a-e777ee8afa3b admin admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:31:12.529678 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8122d67d-71c4-45a8-a66a-e777ee8afa3b admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:31:12.529955 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8122d67d-71c4-45a8-a66a-e777ee8afa3b admin admin] PATH_INFO: `/v3/roles` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:31:12.530706 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8122d67d-71c4-45a8-a66a-e777ee8afa3b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:31:12.530813 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73cd49a7-59f8-4a7b-bcbb-cdbbaad63ff4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:31:12.531478 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cd49a7-59f8-4a7b-bcbb-cdbbaad63ff4 admin admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:31:12.531763 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cd49a7-59f8-4a7b-bcbb-cdbbaad63ff4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:31:12.532032 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cd49a7-59f8-4a7b-bcbb-cdbbaad63ff4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:31:12.532364 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8122d67d-71c4-45a8-a66a-e777ee8afa3b admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:31:12.532786 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73cd49a7-59f8-4a7b-bcbb-cdbbaad63ff4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:31:12.534624 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73cd49a7-59f8-4a7b-bcbb-cdbbaad63ff4 admin admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:31:12.542696 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 795/1592] 10.222.0.49 () {58 vars in 1121 bytes} [Wed Jun 15 10:31:12 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 10:31:12.543486 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8122d67d-71c4-45a8-a66a-e777ee8afa3b admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:31:12.547702 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 798/1593] 10.222.0.49 () {58 vars in 1121 bytes} [Wed Jun 15 10:31:12 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 10:31:12.547702 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73cd49a7-59f8-4a7b-bcbb-cdbbaad63ff4 admin admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:31:12.554366 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8122d67d-71c4-45a8-a66a-e777ee8afa3b admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:31:12.554660 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73cd49a7-59f8-4a7b-bcbb-cdbbaad63ff4 admin admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:31:12.559494 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4149c90e-c97e-4a6e-ac9a-34e0fbc06d01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:31:12.559845 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3937f295-376b-462a-8515-84a696e426ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:31:12.560139 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4149c90e-c97e-4a6e-ac9a-34e0fbc06d01 admin admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:31:12.560424 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4149c90e-c97e-4a6e-ac9a-34e0fbc06d01 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:31:12.560701 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4149c90e-c97e-4a6e-ac9a-34e0fbc06d01 admin admin] PATH_INFO: `/v3/roles` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:31:12.560810 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3937f295-376b-462a-8515-84a696e426ba admin admin] REQUEST_METHOD: `PUT` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:31:12.560980 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3937f295-376b-462a-8515-84a696e426ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:31:12.561300 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3937f295-376b-462a-8515-84a696e426ba admin admin] PATH_INFO: `/v3/projects/63632453a279433b97469eac2f27b36a/users/a0652c42307246b49ee7728aab8c4ba9/roles/6876e57cc51c448097ab109d35b2a3da` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:31:12.561405 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4149c90e-c97e-4a6e-ac9a-34e0fbc06d01 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:31:12.562971 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4149c90e-c97e-4a6e-ac9a-34e0fbc06d01 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:31:12.573756 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 796/1594] 10.222.0.49 () {58 vars in 1121 bytes} [Wed Jun 15 10:31:12 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 10:31:12.574442 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4149c90e-c97e-4a6e-ac9a-34e0fbc06d01 admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:31:12.579368 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3937f295-376b-462a-8515-84a696e426ba admin admin] RBAC: Authorizing `identity:create_grant(project_id=63632453a279433b97469eac2f27b36a, user_id=a0652c42307246b49ee7728aab8c4ba9, role_id=6876e57cc51c448097ab109d35b2a3da)` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:31:12.580406 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: WARNING py.warnings [None req-3937f295-376b-462a-8515-84a696e426ba admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 15 10:31:12.580406 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: warnings.warn(msg) Jun 15 10:31:12.580406 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: Jun 15 10:31:12.582382 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3937f295-376b-462a-8515-84a696e426ba admin admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:31:12.583601 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4149c90e-c97e-4a6e-ac9a-34e0fbc06d01 admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:31:12.588578 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9379fcd-a185-43d5-bc9b-c20cb48d7299 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:31:12.589229 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9379fcd-a185-43d5-bc9b-c20cb48d7299 admin admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:31:12.589560 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9379fcd-a185-43d5-bc9b-c20cb48d7299 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:31:12.590069 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9379fcd-a185-43d5-bc9b-c20cb48d7299 admin admin] PATH_INFO: `/v3/roles` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:31:12.591271 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9379fcd-a185-43d5-bc9b-c20cb48d7299 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:31:12.593850 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9379fcd-a185-43d5-bc9b-c20cb48d7299 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:31:12.597067 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 799/1595] 10.222.0.49 () {60 vars in 1482 bytes} [Wed Jun 15 10:31:12 2022] PUT /identity/v3/projects/63632453a279433b97469eac2f27b36a/users/a0652c42307246b49ee7728aab8c4ba9/roles/6876e57cc51c448097ab109d35b2a3da => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:31:12.609070 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3937f295-376b-462a-8515-84a696e426ba admin admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:31:12.609172 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 797/1596] 10.222.0.49 () {58 vars in 1121 bytes} [Wed Jun 15 10:31:12 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 10:31:12.609172 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebbf98d-83d9-475c-8fa1-18fec525dbe3 None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:31:12.609172 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebbf98d-83d9-475c-8fa1-18fec525dbe3 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:31:12.609172 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebbf98d-83d9-475c-8fa1-18fec525dbe3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:31:12.624724 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-bebbf98d-83d9-475c-8fa1-18fec525dbe3 None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:31:12.625946 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-bebbf98d-83d9-475c-8fa1-18fec525dbe3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:31:12.634967 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:INSERT=8,keystone:SELECT=92 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:31:12.650066 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:31:12.669006 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3937f295-376b-462a-8515-84a696e426ba admin admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:31:12.673744 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec64c4b-bc55-4aa8-91b3-d52f79c6dd88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:31:12.674633 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec64c4b-bc55-4aa8-91b3-d52f79c6dd88 admin admin] REQUEST_METHOD: `PUT` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:31:12.674919 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec64c4b-bc55-4aa8-91b3-d52f79c6dd88 admin admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:31:12.675229 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec64c4b-bc55-4aa8-91b3-d52f79c6dd88 admin admin] PATH_INFO: `/v3/projects/44fccf2eb3e14011b997d90302f4e59f/users/64b81127d41e46a18a1becdab4b07752/roles/6876e57cc51c448097ab109d35b2a3da` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:31:12.686930 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 798/1597] 10.222.0.49 () {58 vars in 957 bytes} [Wed Jun 15 10:31:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:31:12.687807 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebbf98d-83d9-475c-8fa1-18fec525dbe3 None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:31:12.689121 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-bebbf98d-83d9-475c-8fa1-18fec525dbe3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:31:12.691393 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec64c4b-bc55-4aa8-91b3-d52f79c6dd88 admin admin] RBAC: Authorizing `identity:create_grant(project_id=44fccf2eb3e14011b997d90302f4e59f, user_id=64b81127d41e46a18a1becdab4b07752, role_id=6876e57cc51c448097ab109d35b2a3da)` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:31:12.694240 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec64c4b-bc55-4aa8-91b3-d52f79c6dd88 admin admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:31:12.709709 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 800/1598] 10.222.0.49 () {60 vars in 1482 bytes} [Wed Jun 15 10:31:12 2022] PUT /identity/v3/projects/44fccf2eb3e14011b997d90302f4e59f/users/64b81127d41e46a18a1becdab4b07752/roles/6876e57cc51c448097ab109d35b2a3da => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:31:12.710559 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec64c4b-bc55-4aa8-91b3-d52f79c6dd88 admin admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:31:12.759282 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebbf98d-83d9-475c-8fa1-18fec525dbe3 None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:31:12.778482 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec64c4b-bc55-4aa8-91b3-d52f79c6dd88 admin admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:31:12.784030 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90684ad-7772-4752-b3b2-8cbd788ea5df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:31:12.784713 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90684ad-7772-4752-b3b2-8cbd788ea5df admin admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:31:12.785000 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90684ad-7772-4752-b3b2-8cbd788ea5df admin admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:31:12.785278 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90684ad-7772-4752-b3b2-8cbd788ea5df admin admin] PATH_INFO: `/v3/roles` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:31:12.786003 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d90684ad-7772-4752-b3b2-8cbd788ea5df admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:31:12.787651 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d90684ad-7772-4752-b3b2-8cbd788ea5df admin admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:31:12.798201 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 801/1599] 10.222.0.49 () {58 vars in 1121 bytes} [Wed Jun 15 10:31:12 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 10:31:12.799010 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90684ad-7772-4752-b3b2-8cbd788ea5df admin admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:31:12.808467 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90684ad-7772-4752-b3b2-8cbd788ea5df admin admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:31:12.813033 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5baffc6-4c62-42d4-8b49-8316027b81b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:31:12.813903 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5baffc6-4c62-42d4-8b49-8316027b81b0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:31:12.814180 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5baffc6-4c62-42d4-8b49-8316027b81b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:31:12.814449 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5baffc6-4c62-42d4-8b49-8316027b81b0 admin admin] PATH_INFO: `/v3/projects/b683b2d399b54484aa5820537038b8af/users/14b720a2d0854d218272390205887100/roles/6876e57cc51c448097ab109d35b2a3da` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:31:12.816863 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3535fe8a-6bd6-4163-85e8-8147bd89143b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0cb44a17d205431793e9a8791e477a32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '559b3e713f5844b6ae37151d56dd419f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:31:12.817514 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3535fe8a-6bd6-4163-85e8-8147bd89143b service neutron] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:31:12.817796 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3535fe8a-6bd6-4163-85e8-8147bd89143b service neutron] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:31:12.818073 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3535fe8a-6bd6-4163-85e8-8147bd89143b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:31:12.827639 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3535fe8a-6bd6-4163-85e8-8147bd89143b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:31:12.829116 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3535fe8a-6bd6-4163-85e8-8147bd89143b service neutron] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:31:12.829756 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5baffc6-4c62-42d4-8b49-8316027b81b0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b683b2d399b54484aa5820537038b8af, user_id=14b720a2d0854d218272390205887100, role_id=6876e57cc51c448097ab109d35b2a3da)` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:31:12.832436 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5baffc6-4c62-42d4-8b49-8316027b81b0 admin admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:31:12.839975 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 799/1600] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Jun 15 10:31:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:31:12.840988 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3535fe8a-6bd6-4163-85e8-8147bd89143b service neutron] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:31:12.846517 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 802/1601] 10.222.0.49 () {60 vars in 1482 bytes} [Wed Jun 15 10:31:12 2022] PUT /identity/v3/projects/b683b2d399b54484aa5820537038b8af/users/14b720a2d0854d218272390205887100/roles/6876e57cc51c448097ab109d35b2a3da => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:31:12.850184 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3535fe8a-6bd6-4163-85e8-8147bd89143b service neutron] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:31:12.851269 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d2f190-f8e9-4790-a3a6-4b22adb85741 None None] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:31:12.851561 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d2f190-f8e9-4790-a3a6-4b22adb85741 None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:31:12.851843 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d2f190-f8e9-4790-a3a6-4b22adb85741 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:31:12.882159 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.auth.core [None req-62d2f190-f8e9-4790-a3a6-4b22adb85741 None None] MFA Rules not processed for user `a0652c42307246b49ee7728aab8c4ba9`. Rule list: `[]` (Enabled: `True`). {{(pid=80119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:31:12.883586 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-62d2f190-f8e9-4790-a3a6-4b22adb85741 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:31:12.913582 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5fd3a60-605d-4349-bdcb-369a4d0c203a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:31:12.914656 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fd3a60-605d-4349-bdcb-369a4d0c203a admin admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:31:12.915040 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fd3a60-605d-4349-bdcb-369a4d0c203a admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:31:12.915447 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fd3a60-605d-4349-bdcb-369a4d0c203a admin admin] PATH_INFO: `/v3/roles` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:31:12.916390 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5fd3a60-605d-4349-bdcb-369a4d0c203a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:31:12.918119 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5fd3a60-605d-4349-bdcb-369a4d0c203a admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:31:12.929138 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 800/1602] 10.222.0.49 () {58 vars in 1121 bytes} [Wed Jun 15 10:31:12 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 10:31:12.932956 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5fd3a60-605d-4349-bdcb-369a4d0c203a admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:31:12.945135 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5fd3a60-605d-4349-bdcb-369a4d0c203a admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:31:12.950386 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24cc4d2e-bdeb-4615-b4e9-8a7121662fe7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:31:12.951062 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cc4d2e-bdeb-4615-b4e9-8a7121662fe7 admin admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:31:12.951377 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cc4d2e-bdeb-4615-b4e9-8a7121662fe7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:31:12.951657 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cc4d2e-bdeb-4615-b4e9-8a7121662fe7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:31:12.952732 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24cc4d2e-bdeb-4615-b4e9-8a7121662fe7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:31:12.953987 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24cc4d2e-bdeb-4615-b4e9-8a7121662fe7 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:31:12.962981 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:31:12.974046 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 803/1603] 10.222.0.49 () {58 vars in 957 bytes} [Wed Jun 15 10:31:12 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:31:12.974046 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d2f190-f8e9-4790-a3a6-4b22adb85741 None None] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:31:12.974697 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 801/1604] 10.222.0.49 () {58 vars in 1121 bytes} [Wed Jun 15 10:31:12 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 10:31:12.974697 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa7d31f-a40e-4622-a550-5dccb8ec9922 None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:31:12.974697 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa7d31f-a40e-4622-a550-5dccb8ec9922 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:31:12.974697 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa7d31f-a40e-4622-a550-5dccb8ec9922 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:31:13.019038 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-5fa7d31f-a40e-4622-a550-5dccb8ec9922 None None] MFA Rules not processed for user `64b81127d41e46a18a1becdab4b07752`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:31:13.020402 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-5fa7d31f-a40e-4622-a550-5dccb8ec9922 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:31:13.030809 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:31:13.065789 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d2f190-f8e9-4790-a3a6-4b22adb85741 None None] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:31:13.069838 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9af618-3e3b-4c2f-bf9e-9a31cb0fef91 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0cb44a17d205431793e9a8791e477a32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '559b3e713f5844b6ae37151d56dd419f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:31:13.070441 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9af618-3e3b-4c2f-bf9e-9a31cb0fef91 service neutron] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:31:13.070806 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9af618-3e3b-4c2f-bf9e-9a31cb0fef91 service neutron] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:31:13.071309 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9af618-3e3b-4c2f-bf9e-9a31cb0fef91 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:31:13.081339 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9af618-3e3b-4c2f-bf9e-9a31cb0fef91 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:31:13.082796 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9af618-3e3b-4c2f-bf9e-9a31cb0fef91 service neutron] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:31:13.093392 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 804/1605] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Jun 15 10:31:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:31:13.094202 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9af618-3e3b-4c2f-bf9e-9a31cb0fef91 service neutron] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:31:13.101066 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 802/1606] 10.222.0.49 () {58 vars in 957 bytes} [Wed Jun 15 10:31:12 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:31:13.102386 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bbd17b-7f75-4c18-8653-095a16758b52 None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:31:13.102675 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bbd17b-7f75-4c18-8653-095a16758b52 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:31:13.102962 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bbd17b-7f75-4c18-8653-095a16758b52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:31:13.109467 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9af618-3e3b-4c2f-bf9e-9a31cb0fef91 service neutron] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:31:13.115905 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca73ecb2-d30d-4ded-848c-4959cc022743 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:31:13.118447 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca73ecb2-d30d-4ded-848c-4959cc022743 admin admin] REQUEST_METHOD: `PUT` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:31:13.119242 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca73ecb2-d30d-4ded-848c-4959cc022743 admin admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:31:13.119832 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca73ecb2-d30d-4ded-848c-4959cc022743 admin admin] PATH_INFO: `/v3/projects/8b42eec6ead74457af76efc70244987c/users/f445676a7f1846acb2964c6a5c7ee0a0/roles/6876e57cc51c448097ab109d35b2a3da` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:31:13.125517 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-09bbd17b-7f75-4c18-8653-095a16758b52 None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:31:13.126817 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-09bbd17b-7f75-4c18-8653-095a16758b52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:31:13.137131 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 803/1607] 10.222.0.49 () {58 vars in 957 bytes} [Wed Jun 15 10:31:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:31:13.138261 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09bbd17b-7f75-4c18-8653-095a16758b52 None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:31:13.143024 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca73ecb2-d30d-4ded-848c-4959cc022743 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8b42eec6ead74457af76efc70244987c, user_id=f445676a7f1846acb2964c6a5c7ee0a0, role_id=6876e57cc51c448097ab109d35b2a3da)` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:31:13.146448 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca73ecb2-d30d-4ded-848c-4959cc022743 admin admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:31:13.151373 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09bbd17b-7f75-4c18-8653-095a16758b52 None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:31:13.158195 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609bfc5b-7cbd-4a83-a7d2-57b52d71fbea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:31:13.159727 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609bfc5b-7cbd-4a83-a7d2-57b52d71fbea admin admin] REQUEST_METHOD: `PUT` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:31:13.160228 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609bfc5b-7cbd-4a83-a7d2-57b52d71fbea admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:31:13.160697 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609bfc5b-7cbd-4a83-a7d2-57b52d71fbea admin admin] PATH_INFO: `/v3/projects/8ec2b87e615e4b09b3b20ef149260f11/users/e0b486907a6644f59a5cb2ab016bc4a1/roles/6876e57cc51c448097ab109d35b2a3da` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:31:13.174066 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 805/1608] 10.222.0.49 () {60 vars in 1482 bytes} [Wed Jun 15 10:31:13 2022] PUT /identity/v3/projects/8b42eec6ead74457af76efc70244987c/users/f445676a7f1846acb2964c6a5c7ee0a0/roles/6876e57cc51c448097ab109d35b2a3da => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:31:13.174066 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca73ecb2-d30d-4ded-848c-4959cc022743 admin admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:31:13.180518 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-609bfc5b-7cbd-4a83-a7d2-57b52d71fbea admin admin] RBAC: Authorizing `identity:create_grant(project_id=8ec2b87e615e4b09b3b20ef149260f11, user_id=e0b486907a6644f59a5cb2ab016bc4a1, role_id=6876e57cc51c448097ab109d35b2a3da)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:31:13.184445 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-609bfc5b-7cbd-4a83-a7d2-57b52d71fbea admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:31:13.207179 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 804/1609] 10.222.0.49 () {60 vars in 1482 bytes} [Wed Jun 15 10:31:13 2022] PUT /identity/v3/projects/8ec2b87e615e4b09b3b20ef149260f11/users/e0b486907a6644f59a5cb2ab016bc4a1/roles/6876e57cc51c448097ab109d35b2a3da => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:31:13.210443 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bba0492-d4ee-4f64-8fde-b5b27780a513 None None] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:31:13.210838 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bba0492-d4ee-4f64-8fde-b5b27780a513 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:31:13.211366 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bba0492-d4ee-4f64-8fde-b5b27780a513 None None] PATH_INFO: `/` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:31:13.220281 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 805/1610] 10.222.0.49 () {54 vars in 961 bytes} [Wed Jun 15 10:31:13 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:31:13.221390 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bba0492-d4ee-4f64-8fde-b5b27780a513 None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:31:13.265217 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca73ecb2-d30d-4ded-848c-4959cc022743 admin admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:31:13.339607 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c5f1b8-2746-409c-8de9-b7964e911b4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:31:13.340709 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c5f1b8-2746-409c-8de9-b7964e911b4d admin admin] REQUEST_METHOD: `PUT` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:31:13.341065 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c5f1b8-2746-409c-8de9-b7964e911b4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:31:13.341409 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c5f1b8-2746-409c-8de9-b7964e911b4d admin admin] PATH_INFO: `/v3/projects/5dc28af73e82414d8f709e447286a962/users/cca7cb5e7620446ebae91b1fe12bbd24/roles/6876e57cc51c448097ab109d35b2a3da` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:31:13.363364 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c5f1b8-2746-409c-8de9-b7964e911b4d admin admin] RBAC: Authorizing `identity:create_grant(project_id=5dc28af73e82414d8f709e447286a962, user_id=cca7cb5e7620446ebae91b1fe12bbd24, role_id=6876e57cc51c448097ab109d35b2a3da)` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:31:13.366620 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c5f1b8-2746-409c-8de9-b7964e911b4d admin admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:31:13.369957 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bba0492-d4ee-4f64-8fde-b5b27780a513 None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:31:13.383714 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78600a28-b09a-4c3a-bf12-18482e3c318e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:31:13.386864 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78600a28-b09a-4c3a-bf12-18482e3c318e admin admin] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:31:13.387200 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78600a28-b09a-4c3a-bf12-18482e3c318e admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:31:13.387684 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78600a28-b09a-4c3a-bf12-18482e3c318e admin admin] PATH_INFO: `/v3/projects` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:31:13.389226 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78600a28-b09a-4c3a-bf12-18482e3c318e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:31:13.393859 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78600a28-b09a-4c3a-bf12-18482e3c318e admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:31:13.394620 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 806/1611] 10.222.0.49 () {60 vars in 1482 bytes} [Wed Jun 15 10:31:13 2022] PUT /identity/v3/projects/5dc28af73e82414d8f709e447286a962/users/cca7cb5e7620446ebae91b1fe12bbd24/roles/6876e57cc51c448097ab109d35b2a3da => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:31:13.396791 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c5f1b8-2746-409c-8de9-b7964e911b4d admin admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:31:13.449230 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 806/1612] 10.222.0.49 () {60 vars in 1152 bytes} [Wed Jun 15 10:31:13 2022] POST /identity/v3/projects => generated 365 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 10:31:13.449230 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18c6bc7-2c9e-4c19-a210-8e1fd521e934 None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:31:13.449230 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18c6bc7-2c9e-4c19-a210-8e1fd521e934 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:31:13.449230 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18c6bc7-2c9e-4c19-a210-8e1fd521e934 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:31:13.466352 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:31:13.484646 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c5f1b8-2746-409c-8de9-b7964e911b4d admin admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:31:13.486604 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-f18c6bc7-2c9e-4c19-a210-8e1fd521e934 None None] MFA Rules not processed for user `f445676a7f1846acb2964c6a5c7ee0a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:31:13.488712 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-f18c6bc7-2c9e-4c19-a210-8e1fd521e934 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:31:13.535415 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:31:13.574121 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03096c73-3577-4776-b5ea-86996bee35af service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0cb44a17d205431793e9a8791e477a32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '559b3e713f5844b6ae37151d56dd419f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:31:13.574464 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03096c73-3577-4776-b5ea-86996bee35af service neutron] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:31:13.574887 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03096c73-3577-4776-b5ea-86996bee35af service neutron] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:31:13.575416 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03096c73-3577-4776-b5ea-86996bee35af service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:31:13.604094 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 807/1613] 10.222.0.49 () {58 vars in 957 bytes} [Wed Jun 15 10:31:13 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:31:13.605568 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b605e5-d976-4bd8-a84d-3ea29a51f5fd None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:31:13.605873 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b605e5-d976-4bd8-a84d-3ea29a51f5fd None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:31:13.606155 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b605e5-d976-4bd8-a84d-3ea29a51f5fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:31:13.665324 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-a1b605e5-d976-4bd8-a84d-3ea29a51f5fd None None] MFA Rules not processed for user `e0b486907a6644f59a5cb2ab016bc4a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:31:13.667586 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-a1b605e5-d976-4bd8-a84d-3ea29a51f5fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:31:13.698120 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03096c73-3577-4776-b5ea-86996bee35af service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:31:13.699685 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03096c73-3577-4776-b5ea-86996bee35af service neutron] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:31:13.711071 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 807/1614] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Jun 15 10:31:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:31:13.712484 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e781bf-6ccc-45dd-899a-41209f1af38c None None] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:31:13.712888 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e781bf-6ccc-45dd-899a-41209f1af38c None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:31:13.713688 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e781bf-6ccc-45dd-899a-41209f1af38c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:31:13.736036 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.auth.core [None req-77e781bf-6ccc-45dd-899a-41209f1af38c None None] MFA Rules not processed for user `5ac54528370d4747b6e8777fba9d0c82`. Rule list: `[]` (Enabled: `True`). {{(pid=80119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:31:13.737457 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-77e781bf-6ccc-45dd-899a-41209f1af38c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:31:13.773808 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 808/1615] 10.222.0.49 () {58 vars in 957 bytes} [Wed Jun 15 10:31:13 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:31:13.776361 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e87958-215b-4397-850f-ad30c21e4b13 None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:31:13.776942 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e87958-215b-4397-850f-ad30c21e4b13 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:31:13.777567 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e87958-215b-4397-850f-ad30c21e4b13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:31:13.801753 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-99e87958-215b-4397-850f-ad30c21e4b13 None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:31:13.804763 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-99e87958-215b-4397-850f-ad30c21e4b13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:31:13.833829 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 809/1616] 10.222.0.49 () {58 vars in 957 bytes} [Wed Jun 15 10:31:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:31:13.835266 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e87958-215b-4397-850f-ad30c21e4b13 None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:31:13.849525 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e87958-215b-4397-850f-ad30c21e4b13 None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:31:13.856971 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc53c6b-48a7-4ffc-bd99-a51eeb9e417c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:31:13.858043 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc53c6b-48a7-4ffc-bd99-a51eeb9e417c admin admin] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:31:13.858921 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc53c6b-48a7-4ffc-bd99-a51eeb9e417c admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:31:13.859850 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc53c6b-48a7-4ffc-bd99-a51eeb9e417c admin admin] PATH_INFO: `/v3/users` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:31:13.861007 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc53c6b-48a7-4ffc-bd99-a51eeb9e417c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:31:13.863078 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc53c6b-48a7-4ffc-bd99-a51eeb9e417c admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:31:13.892498 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 810/1617] 10.222.0.49 () {60 vars in 1143 bytes} [Wed Jun 15 10:31:13 2022] POST /identity/v3/users => generated 289 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 10:31:13.893466 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc53c6b-48a7-4ffc-bd99-a51eeb9e417c admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:31:13.899891 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 808/1618] 10.222.0.49 () {58 vars in 1058 bytes} [Wed Jun 15 10:31:13 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:31:13.901209 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e781bf-6ccc-45dd-899a-41209f1af38c None None] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:31:13.924015 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e781bf-6ccc-45dd-899a-41209f1af38c None None] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:31:13.931546 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8b8da0-e09f-4538-8678-20856194853b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:31:13.933227 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8b8da0-e09f-4538-8678-20856194853b admin admin] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:31:13.933843 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8b8da0-e09f-4538-8678-20856194853b admin admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:31:13.934398 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8b8da0-e09f-4538-8678-20856194853b admin admin] PATH_INFO: `/v3/projects` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:31:13.935388 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e8b8da0-e09f-4538-8678-20856194853b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:31:13.947607 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e8b8da0-e09f-4538-8678-20856194853b admin admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:31:13.995431 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc53c6b-48a7-4ffc-bd99-a51eeb9e417c admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:31:14.007607 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3981173-a50d-4af8-b00d-32020dbbcd2e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20d1327553404b80aecee87c78fcb52b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '559b3e713f5844b6ae37151d56dd419f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:31:14.008692 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3981173-a50d-4af8-b00d-32020dbbcd2e service cinder] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:31:14.010028 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3981173-a50d-4af8-b00d-32020dbbcd2e service cinder] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:31:14.010889 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3981173-a50d-4af8-b00d-32020dbbcd2e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:31:14.013700 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 809/1619] 10.222.0.49 () {60 vars in 1152 bytes} [Wed Jun 15 10:31:13 2022] POST /identity/v3/projects => generated 385 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 10:31:14.017881 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8b8da0-e09f-4538-8678-20856194853b admin admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:31:14.033256 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3981173-a50d-4af8-b00d-32020dbbcd2e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:31:14.037541 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3981173-a50d-4af8-b00d-32020dbbcd2e service cinder] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:31:14.049212 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:31:14.073889 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 811/1620] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Jun 15 10:31:13 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 181 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:31:14.078331 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3981173-a50d-4af8-b00d-32020dbbcd2e service cinder] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:31:14.103351 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3981173-a50d-4af8-b00d-32020dbbcd2e service cinder] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:31:14.161346 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8b8da0-e09f-4538-8678-20856194853b admin admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:31:14.170880 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85952093-c07a-4bc0-9058-8e3439471a5d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0cb44a17d205431793e9a8791e477a32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '559b3e713f5844b6ae37151d56dd419f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:31:14.171598 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85952093-c07a-4bc0-9058-8e3439471a5d service neutron] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:31:14.171880 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85952093-c07a-4bc0-9058-8e3439471a5d service neutron] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:31:14.172160 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85952093-c07a-4bc0-9058-8e3439471a5d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:31:14.186275 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:31:14.225652 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e19ca1-561e-408d-9f6c-d27c56627b22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:31:14.226472 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e19ca1-561e-408d-9f6c-d27c56627b22 admin admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:31:14.226655 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e19ca1-561e-408d-9f6c-d27c56627b22 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:31:14.226872 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e19ca1-561e-408d-9f6c-d27c56627b22 admin admin] PATH_INFO: `/v3/roles` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:31:14.227639 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e19ca1-561e-408d-9f6c-d27c56627b22 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:31:14.230120 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e19ca1-561e-408d-9f6c-d27c56627b22 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:31:14.253698 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 812/1621] 10.222.0.49 () {58 vars in 1121 bytes} [Wed Jun 15 10:31:14 2022] GET /identity/v3/roles => generated 1420 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 10:31:14.255566 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e19ca1-561e-408d-9f6c-d27c56627b22 admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:31:14.315584 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85952093-c07a-4bc0-9058-8e3439471a5d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:31:14.317471 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85952093-c07a-4bc0-9058-8e3439471a5d service neutron] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:31:14.338261 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 810/1622] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Jun 15 10:31:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:31:14.340294 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85952093-c07a-4bc0-9058-8e3439471a5d service neutron] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:31:14.366219 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85952093-c07a-4bc0-9058-8e3439471a5d service neutron] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:31:14.379637 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e19ca1-561e-408d-9f6c-d27c56627b22 admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:31:14.382045 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c945e1-9991-4574-b6b3-49bbabfaa9c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:31:14.382922 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c945e1-9991-4574-b6b3-49bbabfaa9c2 admin admin] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:31:14.383238 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c945e1-9991-4574-b6b3-49bbabfaa9c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:31:14.384833 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c945e1-9991-4574-b6b3-49bbabfaa9c2 admin admin] PATH_INFO: `/v3/users` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:31:14.385554 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c945e1-9991-4574-b6b3-49bbabfaa9c2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:31:14.390981 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75371e93-4c69-408e-b662-b95732a52a1e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ac54528370d4747b6e8777fba9d0c82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '559b3e713f5844b6ae37151d56dd419f', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:31:14.391825 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c945e1-9991-4574-b6b3-49bbabfaa9c2 admin admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:31:14.395660 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75371e93-4c69-408e-b662-b95732a52a1e service nova] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:31:14.396509 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75371e93-4c69-408e-b662-b95732a52a1e service nova] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:31:14.396903 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75371e93-4c69-408e-b662-b95732a52a1e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:31:14.442216 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 811/1623] 10.222.0.49 () {60 vars in 1143 bytes} [Wed Jun 15 10:31:14 2022] POST /identity/v3/users => generated 299 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 10:31:14.442216 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c945e1-9991-4574-b6b3-49bbabfaa9c2 admin admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:31:14.469507 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c945e1-9991-4574-b6b3-49bbabfaa9c2 admin admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:31:14.474687 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19fd7175-4c4b-4a6b-b87f-622be32e16c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:31:14.475362 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fd7175-4c4b-4a6b-b87f-622be32e16c1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:31:14.475645 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fd7175-4c4b-4a6b-b87f-622be32e16c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:31:14.475923 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fd7175-4c4b-4a6b-b87f-622be32e16c1 admin admin] PATH_INFO: `/v3/projects/888edbc1e4ba47debf36ca4f85b4f446/users/7c71258089f34535b5dbb17a781b3115/roles/8051174e7a7947d5b991ebd793c2e543` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:31:14.515600 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19fd7175-4c4b-4a6b-b87f-622be32e16c1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=888edbc1e4ba47debf36ca4f85b4f446, user_id=7c71258089f34535b5dbb17a781b3115, role_id=8051174e7a7947d5b991ebd793c2e543)` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:31:14.518545 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19fd7175-4c4b-4a6b-b87f-622be32e16c1 admin admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:31:14.530558 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75371e93-4c69-408e-b662-b95732a52a1e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:31:14.535736 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75371e93-4c69-408e-b662-b95732a52a1e service nova] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:31:14.545735 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 812/1624] 10.222.0.49 () {60 vars in 1482 bytes} [Wed Jun 15 10:31:14 2022] PUT /identity/v3/projects/888edbc1e4ba47debf36ca4f85b4f446/users/7c71258089f34535b5dbb17a781b3115/roles/8051174e7a7947d5b991ebd793c2e543 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:31:14.546833 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19fd7175-4c4b-4a6b-b87f-622be32e16c1 admin admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:31:14.559929 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 813/1625] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Jun 15 10:31:14 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:31:14.561335 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30299bb6-3a14-4a6e-b211-558fe4d88786 None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:31:14.565899 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30299bb6-3a14-4a6e-b211-558fe4d88786 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:31:14.566200 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30299bb6-3a14-4a6e-b211-558fe4d88786 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:31:14.639107 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-30299bb6-3a14-4a6e-b211-558fe4d88786 None None] MFA Rules not processed for user `38e2794bd474425d9d75f21c1929972b`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:31:14.642290 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-30299bb6-3a14-4a6e-b211-558fe4d88786 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:31:14.662775 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19fd7175-4c4b-4a6b-b87f-622be32e16c1 admin admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:31:14.671412 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5bc756c-7a22-4dc5-b985-825a96820914 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:31:14.672234 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bc756c-7a22-4dc5-b985-825a96820914 admin admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:31:14.673914 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bc756c-7a22-4dc5-b985-825a96820914 admin admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:31:14.675201 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bc756c-7a22-4dc5-b985-825a96820914 admin admin] PATH_INFO: `/v3/roles` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:31:14.676161 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5bc756c-7a22-4dc5-b985-825a96820914 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:31:14.678328 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5bc756c-7a22-4dc5-b985-825a96820914 admin admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:31:14.703492 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 813/1626] 10.222.0.49 () {58 vars in 1121 bytes} [Wed Jun 15 10:31:14 2022] GET /identity/v3/roles => generated 1420 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 10:31:14.704321 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5bc756c-7a22-4dc5-b985-825a96820914 admin admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:31:14.720674 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5bc756c-7a22-4dc5-b985-825a96820914 admin admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:31:14.734101 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc511d6-51a8-487e-ace9-22d8e7149571 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:31:14.736058 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc511d6-51a8-487e-ace9-22d8e7149571 admin admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:31:14.736340 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc511d6-51a8-487e-ace9-22d8e7149571 admin admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:31:14.736774 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc511d6-51a8-487e-ace9-22d8e7149571 admin admin] PATH_INFO: `/v3/roles` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:31:14.742320 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc511d6-51a8-487e-ace9-22d8e7149571 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:31:14.745153 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc511d6-51a8-487e-ace9-22d8e7149571 admin admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:31:14.777405 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 814/1627] 10.222.0.49 () {58 vars in 1121 bytes} [Wed Jun 15 10:31:14 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 10:31:14.783105 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc511d6-51a8-487e-ace9-22d8e7149571 admin admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:31:14.808718 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 814/1628] 10.222.0.49 () {58 vars in 957 bytes} [Wed Jun 15 10:31:14 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 248 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:31:14.809760 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc511d6-51a8-487e-ace9-22d8e7149571 admin admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:31:14.810083 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30299bb6-3a14-4a6e-b211-558fe4d88786 None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:31:14.825282 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502c9543-3d25-45e9-a191-205761b3d0bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:31:14.825869 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:31:14.826719 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502c9543-3d25-45e9-a191-205761b3d0bf admin admin] REQUEST_METHOD: `PUT` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:31:14.827040 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502c9543-3d25-45e9-a191-205761b3d0bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:31:14.827367 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502c9543-3d25-45e9-a191-205761b3d0bf admin admin] PATH_INFO: `/v3/projects/be949cf4b0504052aad5ff0f70363837/users/62ded95feacf4e56b572c3e9f86891da/roles/8051174e7a7947d5b991ebd793c2e543` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:31:14.831732 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:31:14.832170 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30299bb6-3a14-4a6e-b211-558fe4d88786 None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:31:14.839890 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86298070-a5e9-4218-8e0c-0e21744f48da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:31:14.841049 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86298070-a5e9-4218-8e0c-0e21744f48da admin admin] REQUEST_METHOD: `PUT` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:31:14.841619 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86298070-a5e9-4218-8e0c-0e21744f48da admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:31:14.842062 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86298070-a5e9-4218-8e0c-0e21744f48da admin admin] PATH_INFO: `/v3/projects/888edbc1e4ba47debf36ca4f85b4f446/users/7c71258089f34535b5dbb17a781b3115/roles/8051174e7a7947d5b991ebd793c2e543` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:31:14.844615 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86298070-a5e9-4218-8e0c-0e21744f48da admin admin] RBAC: Authorizing `identity:create_grant(project_id=888edbc1e4ba47debf36ca4f85b4f446, user_id=7c71258089f34535b5dbb17a781b3115, role_id=8051174e7a7947d5b991ebd793c2e543)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:31:14.847520 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86298070-a5e9-4218-8e0c-0e21744f48da admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:31:14.862489 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 815/1629] 10.222.0.49 () {60 vars in 1482 bytes} [Wed Jun 15 10:31:14 2022] PUT /identity/v3/projects/888edbc1e4ba47debf36ca4f85b4f446/users/7c71258089f34535b5dbb17a781b3115/roles/8051174e7a7947d5b991ebd793c2e543 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:31:14.864765 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66027a1-010c-41e2-93ef-29055040246e None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:31:14.865616 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66027a1-010c-41e2-93ef-29055040246e None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:31:14.866147 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66027a1-010c-41e2-93ef-29055040246e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:31:14.873647 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-502c9543-3d25-45e9-a191-205761b3d0bf admin admin] RBAC: Authorizing `identity:create_grant(project_id=be949cf4b0504052aad5ff0f70363837, user_id=62ded95feacf4e56b572c3e9f86891da, role_id=8051174e7a7947d5b991ebd793c2e543)` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:31:14.879723 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-502c9543-3d25-45e9-a191-205761b3d0bf admin admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:31:14.906955 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 815/1630] 10.222.0.49 () {60 vars in 1482 bytes} [Wed Jun 15 10:31:14 2022] PUT /identity/v3/projects/be949cf4b0504052aad5ff0f70363837/users/62ded95feacf4e56b572c3e9f86891da/roles/8051174e7a7947d5b991ebd793c2e543 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:31:14.908814 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-d66027a1-010c-41e2-93ef-29055040246e None None] MFA Rules not processed for user `38e2794bd474425d9d75f21c1929972b`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:31:14.911592 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-d66027a1-010c-41e2-93ef-29055040246e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:31:14.911757 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7566596-3e33-4ffb-a0bc-df8f23810837 None None] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:31:14.911900 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7566596-3e33-4ffb-a0bc-df8f23810837 None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:31:14.912188 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7566596-3e33-4ffb-a0bc-df8f23810837 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:31:14.989584 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.auth.core [None req-b7566596-3e33-4ffb-a0bc-df8f23810837 None None] MFA Rules not processed for user `7c71258089f34535b5dbb17a781b3115`. Rule list: `[]` (Enabled: `True`). {{(pid=80119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:31:14.990683 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-b7566596-3e33-4ffb-a0bc-df8f23810837 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:31:15.083221 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 816/1631] 10.222.0.49 () {58 vars in 957 bytes} [Wed Jun 15 10:31:14 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:31:15.084263 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66027a1-010c-41e2-93ef-29055040246e None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:31:15.161914 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 816/1632] 10.222.0.49 () {58 vars in 957 bytes} [Wed Jun 15 10:31:14 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 253 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:31:15.164617 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3846d377-8402-4142-8af3-812d4a9ecb96 None None] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:31:15.167598 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3846d377-8402-4142-8af3-812d4a9ecb96 None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:31:15.168012 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3846d377-8402-4142-8af3-812d4a9ecb96 None None] PATH_INFO: `/` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:31:15.178894 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 817/1633] 10.222.0.49 () {54 vars in 912 bytes} [Wed Jun 15 10:31:15 2022] GET /identity => generated 270 bytes in 16 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:31:15.180373 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4187d25f-ece1-4150-a48a-fb5bcf6c4754 None None] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:31:15.180953 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4187d25f-ece1-4150-a48a-fb5bcf6c4754 None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:31:15.183457 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4187d25f-ece1-4150-a48a-fb5bcf6c4754 None None] PATH_INFO: `/` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:31:15.186351 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 818/1634] 10.222.0.49 () {54 vars in 961 bytes} [Wed Jun 15 10:31:15 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:31:15.187263 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4187d25f-ece1-4150-a48a-fb5bcf6c4754 None None] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:31:15.316106 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66027a1-010c-41e2-93ef-29055040246e None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:31:15.339921 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e85dd4-4769-41a4-8c93-4da677eddfed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:31:15.340632 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e85dd4-4769-41a4-8c93-4da677eddfed admin admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:31:15.340919 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e85dd4-4769-41a4-8c93-4da677eddfed admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:31:15.341198 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e85dd4-4769-41a4-8c93-4da677eddfed admin admin] PATH_INFO: `/v3/roles` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:31:15.341944 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e85dd4-4769-41a4-8c93-4da677eddfed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:31:15.343605 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e85dd4-4769-41a4-8c93-4da677eddfed admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:31:15.368990 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 817/1635] 10.222.0.49 () {58 vars in 1121 bytes} [Wed Jun 15 10:31:15 2022] GET /identity/v3/roles => generated 1420 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 10:31:15.368990 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43cf88e-f0d0-42d6-9a60-8196ff7d5b25 None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:31:15.368990 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43cf88e-f0d0-42d6-9a60-8196ff7d5b25 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:31:15.368990 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43cf88e-f0d0-42d6-9a60-8196ff7d5b25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:31:15.383128 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4187d25f-ece1-4150-a48a-fb5bcf6c4754 None None] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:31:15.394533 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e120db21-3d63-4fa1-a38f-b0588619a7df service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20d1327553404b80aecee87c78fcb52b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '559b3e713f5844b6ae37151d56dd419f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:31:15.395401 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e120db21-3d63-4fa1-a38f-b0588619a7df service cinder] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:31:15.398184 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e120db21-3d63-4fa1-a38f-b0588619a7df service cinder] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:31:15.401340 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e120db21-3d63-4fa1-a38f-b0588619a7df service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:31:15.404319 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-f43cf88e-f0d0-42d6-9a60-8196ff7d5b25 None None] MFA Rules not processed for user `5ac54528370d4747b6e8777fba9d0c82`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:31:15.405685 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-f43cf88e-f0d0-42d6-9a60-8196ff7d5b25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:31:15.422643 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e120db21-3d63-4fa1-a38f-b0588619a7df service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:31:15.427780 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e120db21-3d63-4fa1-a38f-b0588619a7df service cinder] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:31:15.456005 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 819/1636] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Jun 15 10:31:15 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 269 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:31:15.459129 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c46486-d787-485c-a1ad-c4c89abc194f None None] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:31:15.463367 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c46486-d787-485c-a1ad-c4c89abc194f None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:31:15.463767 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c46486-d787-485c-a1ad-c4c89abc194f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:31:15.505444 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.auth.core [None req-20c46486-d787-485c-a1ad-c4c89abc194f None None] MFA Rules not processed for user `5ac54528370d4747b6e8777fba9d0c82`. Rule list: `[]` (Enabled: `True`). {{(pid=80119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:31:15.507180 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-20c46486-d787-485c-a1ad-c4c89abc194f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:31:15.542457 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 820/1637] 10.222.0.49 () {58 vars in 1058 bytes} [Wed Jun 15 10:31:15 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:31:15.550127 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c46486-d787-485c-a1ad-c4c89abc194f None None] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:31:15.556436 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 818/1638] 10.222.0.49 () {58 vars in 1009 bytes} [Wed Jun 15 10:31:15 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:31:15.563486 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43cf88e-f0d0-42d6-9a60-8196ff7d5b25 None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:31:15.582612 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c46486-d787-485c-a1ad-c4c89abc194f None None] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:31:15.587186 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43cf88e-f0d0-42d6-9a60-8196ff7d5b25 None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:31:15.597705 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147b6cb7-4950-41e9-a732-5a4337752d77 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ac54528370d4747b6e8777fba9d0c82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '559b3e713f5844b6ae37151d56dd419f', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:31:15.598075 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e297662a-d21a-4b9a-bee3-2afd3e103dbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:31:15.599195 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e297662a-d21a-4b9a-bee3-2afd3e103dbe admin admin] REQUEST_METHOD: `PUT` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:31:15.599589 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e297662a-d21a-4b9a-bee3-2afd3e103dbe admin admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:31:15.599970 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e297662a-d21a-4b9a-bee3-2afd3e103dbe admin admin] PATH_INFO: `/v3/projects/be949cf4b0504052aad5ff0f70363837/users/62ded95feacf4e56b572c3e9f86891da/roles/8051174e7a7947d5b991ebd793c2e543` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:31:15.600279 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147b6cb7-4950-41e9-a732-5a4337752d77 service nova] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:31:15.601289 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147b6cb7-4950-41e9-a732-5a4337752d77 service nova] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:31:15.602916 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e297662a-d21a-4b9a-bee3-2afd3e103dbe admin admin] RBAC: Authorizing `identity:create_grant(project_id=be949cf4b0504052aad5ff0f70363837, user_id=62ded95feacf4e56b572c3e9f86891da, role_id=8051174e7a7947d5b991ebd793c2e543)` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:31:15.604439 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147b6cb7-4950-41e9-a732-5a4337752d77 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:31:15.608215 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e297662a-d21a-4b9a-bee3-2afd3e103dbe admin admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:31:15.621544 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-147b6cb7-4950-41e9-a732-5a4337752d77 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:31:15.623503 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-147b6cb7-4950-41e9-a732-5a4337752d77 service nova] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:31:15.633064 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 821/1639] 10.222.0.49 () {60 vars in 1482 bytes} [Wed Jun 15 10:31:15 2022] PUT /identity/v3/projects/be949cf4b0504052aad5ff0f70363837/users/62ded95feacf4e56b572c3e9f86891da/roles/8051174e7a7947d5b991ebd793c2e543 => generated 0 bytes in 90 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:31:15.643669 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e297662a-d21a-4b9a-bee3-2afd3e103dbe admin admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:31:15.662353 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 819/1640] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Jun 15 10:31:15 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:31:15.667505 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ec06eb-d811-4d30-86e4-464d44ba1b91 None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:31:15.668092 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ec06eb-d811-4d30-86e4-464d44ba1b91 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:31:15.668580 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ec06eb-d811-4d30-86e4-464d44ba1b91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:31:15.704880 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:31:15.737606 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-61ec06eb-d811-4d30-86e4-464d44ba1b91 None None] MFA Rules not processed for user `62ded95feacf4e56b572c3e9f86891da`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:31:15.739430 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-61ec06eb-d811-4d30-86e4-464d44ba1b91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:31:15.762251 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e297662a-d21a-4b9a-bee3-2afd3e103dbe admin admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:31:15.776321 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc1722f-00ab-47ab-894a-040968100397 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20d1327553404b80aecee87c78fcb52b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '559b3e713f5844b6ae37151d56dd419f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:31:15.777357 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc1722f-00ab-47ab-894a-040968100397 service cinder] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:31:15.777920 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc1722f-00ab-47ab-894a-040968100397 service cinder] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:31:15.777920 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc1722f-00ab-47ab-894a-040968100397 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:31:15.793169 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:31:15.912078 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 820/1641] 10.222.0.49 () {58 vars in 957 bytes} [Wed Jun 15 10:31:15 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 246 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:31:15.912078 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ec06eb-d811-4d30-86e4-464d44ba1b91 None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:31:15.940542 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ec06eb-d811-4d30-86e4-464d44ba1b91 None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:31:15.958522 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03fe729-ce2b-4421-8a57-24312f2388a4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20d1327553404b80aecee87c78fcb52b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '559b3e713f5844b6ae37151d56dd419f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:31:15.959379 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03fe729-ce2b-4421-8a57-24312f2388a4 service cinder] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:31:15.959768 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03fe729-ce2b-4421-8a57-24312f2388a4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:31:15.960151 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03fe729-ce2b-4421-8a57-24312f2388a4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:31:16.045388 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc1722f-00ab-47ab-894a-040968100397 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:31:16.052091 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc1722f-00ab-47ab-894a-040968100397 service cinder] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:31:16.073118 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c03fe729-ce2b-4421-8a57-24312f2388a4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:31:16.075932 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c03fe729-ce2b-4421-8a57-24312f2388a4 service cinder] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:31:16.092556 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 822/1642] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Jun 15 10:31:15 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 459 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:31:16.101488 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 821/1643] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Jun 15 10:31:15 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 191 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:31:16.268365 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e47e11-2897-4c9e-8adc-a9d74d83f231 None None] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:31:16.269509 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e47e11-2897-4c9e-8adc-a9d74d83f231 None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:31:16.269959 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e47e11-2897-4c9e-8adc-a9d74d83f231 None None] PATH_INFO: `/` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:31:16.272048 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 823/1644] 10.222.0.49 () {54 vars in 912 bytes} [Wed Jun 15 10:31:16 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:31:16.278400 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce886d89-1235-47cc-80fb-08a2e8539826 None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:31:16.278881 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce886d89-1235-47cc-80fb-08a2e8539826 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:31:16.279457 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce886d89-1235-47cc-80fb-08a2e8539826 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:31:16.338417 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-ce886d89-1235-47cc-80fb-08a2e8539826 None None] MFA Rules not processed for user `5ac54528370d4747b6e8777fba9d0c82`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:31:16.344575 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-ce886d89-1235-47cc-80fb-08a2e8539826 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:31:16.389935 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 822/1645] 10.222.0.49 () {58 vars in 1009 bytes} [Wed Jun 15 10:31:16 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:31:16.405171 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e47e11-2897-4c9e-8adc-a9d74d83f231 None None] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:31:16.407351 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-89e47e11-2897-4c9e-8adc-a9d74d83f231 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:31:16.515918 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e47e11-2897-4c9e-8adc-a9d74d83f231 None None] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:31:16.522475 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e3cb52-f438-4944-a177-43db448a7b7a service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91f2fe3be62e49548eae208e57bf2a93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '559b3e713f5844b6ae37151d56dd419f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:31:16.524101 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e3cb52-f438-4944-a177-43db448a7b7a service glance] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:31:16.525295 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e3cb52-f438-4944-a177-43db448a7b7a service glance] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:31:16.525295 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e3cb52-f438-4944-a177-43db448a7b7a service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:31:16.552893 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e3cb52-f438-4944-a177-43db448a7b7a service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:31:16.555528 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e3cb52-f438-4944-a177-43db448a7b7a service glance] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:31:16.588699 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 824/1646] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Jun 15 10:31:16 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 184 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:31:16.614198 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce886d89-1235-47cc-80fb-08a2e8539826 None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:31:16.627995 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce886d89-1235-47cc-80fb-08a2e8539826 None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:31:16.639399 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47892dbf-b582-462f-b33a-0aa7187f509e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20d1327553404b80aecee87c78fcb52b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '559b3e713f5844b6ae37151d56dd419f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:31:16.640919 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47892dbf-b582-462f-b33a-0aa7187f509e service cinder] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:31:16.641729 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47892dbf-b582-462f-b33a-0aa7187f509e service cinder] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:31:16.644735 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47892dbf-b582-462f-b33a-0aa7187f509e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:31:16.679401 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47892dbf-b582-462f-b33a-0aa7187f509e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:31:16.681091 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47892dbf-b582-462f-b33a-0aa7187f509e service cinder] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:31:16.716837 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 823/1647] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Jun 15 10:31:16 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:31:16.951551 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e3cb52-f438-4944-a177-43db448a7b7a service glance] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:31:17.151533 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e3cb52-f438-4944-a177-43db448a7b7a service glance] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:31:17.183720 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e7578a-b791-4348-beaf-21e12207a706 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:31:17.186989 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e7578a-b791-4348-beaf-21e12207a706 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:31:17.188051 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e7578a-b791-4348-beaf-21e12207a706 admin admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:31:17.188347 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e7578a-b791-4348-beaf-21e12207a706 admin admin] PATH_INFO: `/v3/users/e0b486907a6644f59a5cb2ab016bc4a1` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:31:17.190884 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e7578a-b791-4348-beaf-21e12207a706 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e0b486907a6644f59a5cb2ab016bc4a1)` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:31:17.194538 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: WARNING py.warnings [None req-41e7578a-b791-4348-beaf-21e12207a706 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 15 10:31:17.194538 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: warnings.warn(msg) Jun 15 10:31:17.194538 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: Jun 15 10:31:17.198974 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e7578a-b791-4348-beaf-21e12207a706 admin admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:31:17.387816 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:31:17.415569 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-41e7578a-b791-4348-beaf-21e12207a706 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e0b486907a6644f59a5cb2ab016bc4a1'} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:31:17.445163 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-41e7578a-b791-4348-beaf-21e12207a706 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e0b486907a6644f59a5cb2ab016bc4a1'} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:31:17.448092 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-41e7578a-b791-4348-beaf-21e12207a706 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e0b486907a6644f59a5cb2ab016bc4a1'} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:31:17.450569 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-41e7578a-b791-4348-beaf-21e12207a706 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e0b486907a6644f59a5cb2ab016bc4a1'} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:31:17.470859 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-41e7578a-b791-4348-beaf-21e12207a706 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e0b486907a6644f59a5cb2ab016bc4a1'} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:31:17.534179 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 825/1648] 10.222.0.49 () {58 vars in 1223 bytes} [Wed Jun 15 10:31:16 2022] DELETE /identity/v3/users/e0b486907a6644f59a5cb2ab016bc4a1 => generated 0 bytes in 581 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:31:17.537377 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98d6968-8496-468f-bcaf-852681862bc3 None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:31:17.539055 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98d6968-8496-468f-bcaf-852681862bc3 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:31:17.539055 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98d6968-8496-468f-bcaf-852681862bc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:31:17.569658 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-b98d6968-8496-468f-bcaf-852681862bc3 None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:31:17.571055 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-b98d6968-8496-468f-bcaf-852681862bc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:31:17.756274 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 824/1649] 10.222.0.49 () {58 vars in 957 bytes} [Wed Jun 15 10:31:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:31:17.832839 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e7578a-b791-4348-beaf-21e12207a706 admin admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:31:17.834414 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-41e7578a-b791-4348-beaf-21e12207a706 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:31:17.972616 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e7578a-b791-4348-beaf-21e12207a706 admin admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:31:17.986544 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb37e9d-7aa2-4147-947b-8adb7eac8750 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0cb44a17d205431793e9a8791e477a32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '559b3e713f5844b6ae37151d56dd419f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:31:17.991843 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb37e9d-7aa2-4147-947b-8adb7eac8750 service neutron] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:31:17.992304 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb37e9d-7aa2-4147-947b-8adb7eac8750 service neutron] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:31:17.992918 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb37e9d-7aa2-4147-947b-8adb7eac8750 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:31:18.035405 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb37e9d-7aa2-4147-947b-8adb7eac8750 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:31:18.040218 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb37e9d-7aa2-4147-947b-8adb7eac8750 service neutron] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:31:18.095519 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 826/1650] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Jun 15 10:31:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 263 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:31:18.475827 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744c14d1-5a78-465c-8245-f0943718ba31 None None] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:31:18.476386 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744c14d1-5a78-465c-8245-f0943718ba31 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:31:18.476727 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744c14d1-5a78-465c-8245-f0943718ba31 None None] PATH_INFO: `/` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:31:18.486851 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 825/1651] 10.222.0.49 () {54 vars in 912 bytes} [Wed Jun 15 10:31:18 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:31:18.503700 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb96e84-4a8f-4281-bbc5-366b143f7269 None None] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:31:18.504236 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb96e84-4a8f-4281-bbc5-366b143f7269 None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:31:18.504435 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb96e84-4a8f-4281-bbc5-366b143f7269 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:31:18.580054 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.auth.core [None req-1fb96e84-4a8f-4281-bbc5-366b143f7269 None None] MFA Rules not processed for user `0cb44a17d205431793e9a8791e477a32`. Rule list: `[]` (Enabled: `True`). {{(pid=80119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:31:18.581487 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-1fb96e84-4a8f-4281-bbc5-366b143f7269 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:31:18.590718 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 827/1652] 10.222.0.49 () {58 vars in 1009 bytes} [Wed Jun 15 10:31:18 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:31:18.652653 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744c14d1-5a78-465c-8245-f0943718ba31 None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:31:18.656713 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-744c14d1-5a78-465c-8245-f0943718ba31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:31:18.676776 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744c14d1-5a78-465c-8245-f0943718ba31 None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:31:18.684523 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d74c8d3-6453-48ec-bf7c-133ea50a9abc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0cb44a17d205431793e9a8791e477a32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '559b3e713f5844b6ae37151d56dd419f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:31:18.685387 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d74c8d3-6453-48ec-bf7c-133ea50a9abc service neutron] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:31:18.685684 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d74c8d3-6453-48ec-bf7c-133ea50a9abc service neutron] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:31:18.686095 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d74c8d3-6453-48ec-bf7c-133ea50a9abc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:31:18.710397 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d74c8d3-6453-48ec-bf7c-133ea50a9abc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:31:18.712033 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d74c8d3-6453-48ec-bf7c-133ea50a9abc service neutron] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:31:18.722542 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 826/1653] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Jun 15 10:31:18 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 70 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:31:19.127134 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb96e84-4a8f-4281-bbc5-366b143f7269 None None] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:31:19.129069 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-1fb96e84-4a8f-4281-bbc5-366b143f7269 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:31:19.189923 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb96e84-4a8f-4281-bbc5-366b143f7269 None None] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:31:19.205972 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae0f7a4-8c79-40e9-831d-7756abd9666a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:31:19.206885 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae0f7a4-8c79-40e9-831d-7756abd9666a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:31:19.207197 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae0f7a4-8c79-40e9-831d-7756abd9666a admin admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:31:19.207471 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae0f7a4-8c79-40e9-831d-7756abd9666a admin admin] PATH_INFO: `/v3/projects/8ec2b87e615e4b09b3b20ef149260f11` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:31:19.209332 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae0f7a4-8c79-40e9-831d-7756abd9666a admin admin] RBAC: Authorizing `identity:delete_project(project_id=8ec2b87e615e4b09b3b20ef149260f11)` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:31:19.214905 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae0f7a4-8c79-40e9-831d-7756abd9666a admin admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:31:19.351114 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-9ae0f7a4-8c79-40e9-831d-7756abd9666a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8ec2b87e615e4b09b3b20ef149260f11', 'request_id': 'req-9ae0f7a4-8c79-40e9-831d-7756abd9666a'} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:31:19.371218 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-9ae0f7a4-8c79-40e9-831d-7756abd9666a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8ec2b87e615e4b09b3b20ef149260f11', 'request_id': 'req-9ae0f7a4-8c79-40e9-831d-7756abd9666a'} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:31:19.392377 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-9ae0f7a4-8c79-40e9-831d-7756abd9666a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8ec2b87e615e4b09b3b20ef149260f11', 'request_id': 'req-9ae0f7a4-8c79-40e9-831d-7756abd9666a'} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:31:19.409084 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-9ae0f7a4-8c79-40e9-831d-7756abd9666a admin admin] The token cache is being invalidate because project 8ec2b87e615e4b09b3b20ef149260f11 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80119) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 10:31:19.409711 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-9ae0f7a4-8c79-40e9-831d-7756abd9666a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:31:19.410525 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-9ae0f7a4-8c79-40e9-831d-7756abd9666a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:31:19.412339 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 828/1654] 10.222.0.49 () {58 vars in 1232 bytes} [Wed Jun 15 10:31:19 2022] DELETE /identity/v3/projects/8ec2b87e615e4b09b3b20ef149260f11 => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:31:19.416429 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d74c8d3-6453-48ec-bf7c-133ea50a9abc service neutron] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:31:19.424347 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:31:19.442157 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d74c8d3-6453-48ec-bf7c-133ea50a9abc service neutron] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:31:19.631866 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4cfa852-587d-45be-83bf-86d8e6ae635b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:31:19.632788 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cfa852-587d-45be-83bf-86d8e6ae635b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:31:19.633070 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cfa852-587d-45be-83bf-86d8e6ae635b admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:31:19.633344 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cfa852-587d-45be-83bf-86d8e6ae635b admin admin] PATH_INFO: `/v3/users/62ded95feacf4e56b572c3e9f86891da` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:31:19.636825 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4cfa852-587d-45be-83bf-86d8e6ae635b admin admin] RBAC: Authorizing `identity:delete_user(user_id=62ded95feacf4e56b572c3e9f86891da)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:31:19.640796 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4cfa852-587d-45be-83bf-86d8e6ae635b admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:31:19.690719 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae0f7a4-8c79-40e9-831d-7756abd9666a admin admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:31:19.693465 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-9ae0f7a4-8c79-40e9-831d-7756abd9666a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:31:19.723889 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae0f7a4-8c79-40e9-831d-7756abd9666a admin admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:31:19.741057 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e501fc-1113-43a7-8d1a-a34670887f85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:31:19.743590 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e501fc-1113-43a7-8d1a-a34670887f85 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:31:19.745448 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e501fc-1113-43a7-8d1a-a34670887f85 admin admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:31:19.746233 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e501fc-1113-43a7-8d1a-a34670887f85 admin admin] PATH_INFO: `/v3/users/64b81127d41e46a18a1becdab4b07752` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:31:19.754352 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e501fc-1113-43a7-8d1a-a34670887f85 admin admin] RBAC: Authorizing `identity:delete_user(user_id=64b81127d41e46a18a1becdab4b07752)` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:31:19.769928 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-f4cfa852-587d-45be-83bf-86d8e6ae635b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '62ded95feacf4e56b572c3e9f86891da'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:31:19.782373 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e501fc-1113-43a7-8d1a-a34670887f85 admin admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:31:19.788684 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-f4cfa852-587d-45be-83bf-86d8e6ae635b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '62ded95feacf4e56b572c3e9f86891da'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:31:19.792164 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-f4cfa852-587d-45be-83bf-86d8e6ae635b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '62ded95feacf4e56b572c3e9f86891da'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:31:19.794062 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-f4cfa852-587d-45be-83bf-86d8e6ae635b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '62ded95feacf4e56b572c3e9f86891da'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:31:19.818872 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-f4cfa852-587d-45be-83bf-86d8e6ae635b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '62ded95feacf4e56b572c3e9f86891da'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:31:19.850502 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:31:19.879913 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 827/1655] 10.222.0.49 () {58 vars in 1223 bytes} [Wed Jun 15 10:31:19 2022] DELETE /identity/v3/users/62ded95feacf4e56b572c3e9f86891da => generated 0 bytes in 464 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:31:20.009532 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-22e501fc-1113-43a7-8d1a-a34670887f85 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '64b81127d41e46a18a1becdab4b07752'} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:31:20.024850 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-22e501fc-1113-43a7-8d1a-a34670887f85 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '64b81127d41e46a18a1becdab4b07752'} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:31:20.027487 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-22e501fc-1113-43a7-8d1a-a34670887f85 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '64b81127d41e46a18a1becdab4b07752'} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:31:20.029440 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-22e501fc-1113-43a7-8d1a-a34670887f85 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '64b81127d41e46a18a1becdab4b07752'} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:31:20.068154 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-22e501fc-1113-43a7-8d1a-a34670887f85 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '64b81127d41e46a18a1becdab4b07752'} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:31:20.115175 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 829/1656] 10.222.0.49 () {58 vars in 1223 bytes} [Wed Jun 15 10:31:19 2022] DELETE /identity/v3/users/64b81127d41e46a18a1becdab4b07752 => generated 0 bytes in 425 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:31:20.119731 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982d11fa-3fce-41bc-9ccf-e295faa1e8c8 None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:31:20.120753 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982d11fa-3fce-41bc-9ccf-e295faa1e8c8 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:31:20.121045 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982d11fa-3fce-41bc-9ccf-e295faa1e8c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:31:20.158362 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-982d11fa-3fce-41bc-9ccf-e295faa1e8c8 None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:31:20.163378 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-982d11fa-3fce-41bc-9ccf-e295faa1e8c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:31:20.386321 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 828/1657] 10.222.0.49 () {58 vars in 957 bytes} [Wed Jun 15 10:31:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 268 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:31:20.527814 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e501fc-1113-43a7-8d1a-a34670887f85 admin admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:31:20.529347 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-22e501fc-1113-43a7-8d1a-a34670887f85 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:31:20.669496 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e501fc-1113-43a7-8d1a-a34670887f85 admin admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:31:20.678091 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd61ee7-c133-473c-91c4-ac80f895e99f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0cb44a17d205431793e9a8791e477a32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '559b3e713f5844b6ae37151d56dd419f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:31:20.679347 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd61ee7-c133-473c-91c4-ac80f895e99f service neutron] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:31:20.682409 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd61ee7-c133-473c-91c4-ac80f895e99f service neutron] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:31:20.682700 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd61ee7-c133-473c-91c4-ac80f895e99f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:31:20.717233 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd61ee7-c133-473c-91c4-ac80f895e99f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:31:20.718617 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd61ee7-c133-473c-91c4-ac80f895e99f service neutron] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:31:20.750882 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 830/1658] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Jun 15 10:31:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 224 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 10:31:20.868207 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ac974c-804d-4565-b50a-07f2b5cb3312 None None] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:31:20.869242 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ac974c-804d-4565-b50a-07f2b5cb3312 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:31:20.869311 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ac974c-804d-4565-b50a-07f2b5cb3312 None None] PATH_INFO: `/` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:31:20.870741 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 829/1659] 10.222.0.49 () {54 vars in 960 bytes} [Wed Jun 15 10:31:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:31:20.876917 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23a68c7-e606-4da2-a109-62428ee25d27 None None] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:31:20.876917 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23a68c7-e606-4da2-a109-62428ee25d27 None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:31:20.877176 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23a68c7-e606-4da2-a109-62428ee25d27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:31:20.912978 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.auth.core [None req-c23a68c7-e606-4da2-a109-62428ee25d27 None None] MFA Rules not processed for user `dd03612a525644518517069b3fa43522`. Rule list: `[]` (Enabled: `True`). {{(pid=80119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:31:20.915219 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-c23a68c7-e606-4da2-a109-62428ee25d27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:31:21.107783 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 831/1660] 10.222.0.49 () {58 vars in 1057 bytes} [Wed Jun 15 10:31:20 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:31:21.116053 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ac974c-804d-4565-b50a-07f2b5cb3312 None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:31:21.151661 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ac974c-804d-4565-b50a-07f2b5cb3312 None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:31:21.158680 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0cae62-9aac-4018-a1a3-fc8d5f7c71fa service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd03612a525644518517069b3fa43522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '559b3e713f5844b6ae37151d56dd419f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:31:21.159389 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0cae62-9aac-4018-a1a3-fc8d5f7c71fa service placement] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:31:21.159678 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0cae62-9aac-4018-a1a3-fc8d5f7c71fa service placement] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:31:21.159960 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0cae62-9aac-4018-a1a3-fc8d5f7c71fa service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:31:21.167758 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-0c0cae62-9aac-4018-a1a3-fc8d5f7c71fa service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:31:21.190011 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0cae62-9aac-4018-a1a3-fc8d5f7c71fa service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:31:21.191808 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0cae62-9aac-4018-a1a3-fc8d5f7c71fa service placement] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:31:21.232757 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 830/1661] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Jun 15 10:31:21 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:31:21.428745 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23a68c7-e606-4da2-a109-62428ee25d27 None None] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:31:21.430758 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-c23a68c7-e606-4da2-a109-62428ee25d27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:31:21.457797 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23a68c7-e606-4da2-a109-62428ee25d27 None None] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:31:21.475573 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3286853-e63b-49c5-85ef-205e64a3dd12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:31:21.480099 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3286853-e63b-49c5-85ef-205e64a3dd12 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:31:21.480432 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3286853-e63b-49c5-85ef-205e64a3dd12 admin admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:31:21.480722 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3286853-e63b-49c5-85ef-205e64a3dd12 admin admin] PATH_INFO: `/v3/projects/be949cf4b0504052aad5ff0f70363837` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:31:21.484447 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3286853-e63b-49c5-85ef-205e64a3dd12 admin admin] RBAC: Authorizing `identity:delete_project(project_id=be949cf4b0504052aad5ff0f70363837)` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:31:21.488474 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3286853-e63b-49c5-85ef-205e64a3dd12 admin admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:31:21.580184 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:31:21.622079 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-c3286853-e63b-49c5-85ef-205e64a3dd12 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'be949cf4b0504052aad5ff0f70363837', 'request_id': 'req-c3286853-e63b-49c5-85ef-205e64a3dd12'} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:31:21.633865 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-c3286853-e63b-49c5-85ef-205e64a3dd12 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'be949cf4b0504052aad5ff0f70363837', 'request_id': 'req-c3286853-e63b-49c5-85ef-205e64a3dd12'} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:31:21.648441 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-c3286853-e63b-49c5-85ef-205e64a3dd12 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'be949cf4b0504052aad5ff0f70363837', 'request_id': 'req-c3286853-e63b-49c5-85ef-205e64a3dd12'} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:31:21.664911 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-c3286853-e63b-49c5-85ef-205e64a3dd12 admin admin] The token cache is being invalidate because project be949cf4b0504052aad5ff0f70363837 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80119) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 10:31:21.665575 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-c3286853-e63b-49c5-85ef-205e64a3dd12 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:31:21.666831 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-c3286853-e63b-49c5-85ef-205e64a3dd12 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:31:21.668862 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 832/1662] 10.222.0.49 () {58 vars in 1232 bytes} [Wed Jun 15 10:31:21 2022] DELETE /identity/v3/projects/be949cf4b0504052aad5ff0f70363837 => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 10:31:21.673819 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3568b9-06eb-4df5-b1cd-1af26b37ecea None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:31:21.674118 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3568b9-06eb-4df5-b1cd-1af26b37ecea None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:31:21.674427 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3568b9-06eb-4df5-b1cd-1af26b37ecea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:31:21.698241 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-0e3568b9-06eb-4df5-b1cd-1af26b37ecea None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:31:21.700911 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-0e3568b9-06eb-4df5-b1cd-1af26b37ecea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:31:21.710300 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:31:21.848168 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 831/1663] 10.222.0.49 () {58 vars in 957 bytes} [Wed Jun 15 10:31:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:31:21.856890 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9158ea31-4806-43fd-b64c-db65f56bb381 None None] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:31:21.857693 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9158ea31-4806-43fd-b64c-db65f56bb381 None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:31:21.858118 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9158ea31-4806-43fd-b64c-db65f56bb381 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:31:21.901478 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.auth.core [None req-9158ea31-4806-43fd-b64c-db65f56bb381 None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:31:21.903026 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-9158ea31-4806-43fd-b64c-db65f56bb381 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:31:21.920786 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 833/1664] 10.222.0.49 () {58 vars in 957 bytes} [Wed Jun 15 10:31:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 63 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:31:21.924564 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3568b9-06eb-4df5-b1cd-1af26b37ecea None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:31:21.944603 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3568b9-06eb-4df5-b1cd-1af26b37ecea None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:31:21.956020 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86a36e1-1809-42a2-9193-0d12a200a8af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:31:21.956731 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86a36e1-1809-42a2-9193-0d12a200a8af admin admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:31:21.959633 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86a36e1-1809-42a2-9193-0d12a200a8af admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:31:21.960158 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86a36e1-1809-42a2-9193-0d12a200a8af admin admin] PATH_INFO: `/v3/domains` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:31:21.961218 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86a36e1-1809-42a2-9193-0d12a200a8af admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:31:21.966226 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86a36e1-1809-42a2-9193-0d12a200a8af admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:31:21.968816 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86a36e1-1809-42a2-9193-0d12a200a8af admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:31:21.995448 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 832/1665] 10.222.0.49 () {58 vars in 1152 bytes} [Wed Jun 15 10:31:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:31:22.000179 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9158ea31-4806-43fd-b64c-db65f56bb381 None None] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:31:22.019535 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9158ea31-4806-43fd-b64c-db65f56bb381 None None] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:31:22.042755 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13840ad4-c100-4496-9787-ad4909ed1385 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:31:22.043744 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13840ad4-c100-4496-9787-ad4909ed1385 admin admin] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:31:22.043966 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13840ad4-c100-4496-9787-ad4909ed1385 admin admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:31:22.044243 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13840ad4-c100-4496-9787-ad4909ed1385 admin admin] PATH_INFO: `/v3/projects` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:31:22.044967 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13840ad4-c100-4496-9787-ad4909ed1385 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:31:22.050227 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13840ad4-c100-4496-9787-ad4909ed1385 admin admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:31:22.130431 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 834/1666] 10.222.0.49 () {60 vars in 1152 bytes} [Wed Jun 15 10:31:21 2022] POST /identity/v3/projects => generated 383 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 10:31:22.135402 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86a36e1-1809-42a2-9193-0d12a200a8af admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:31:22.151243 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86a36e1-1809-42a2-9193-0d12a200a8af admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:31:22.324794 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463cb227-242b-4510-9ccc-fb41187b8eaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:31:22.326112 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463cb227-242b-4510-9ccc-fb41187b8eaf admin admin] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:31:22.326405 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463cb227-242b-4510-9ccc-fb41187b8eaf admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:31:22.328609 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463cb227-242b-4510-9ccc-fb41187b8eaf admin admin] PATH_INFO: `/v3/users` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:31:22.361226 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463cb227-242b-4510-9ccc-fb41187b8eaf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:31:22.363399 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463cb227-242b-4510-9ccc-fb41187b8eaf admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:31:22.444439 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2120a9-836d-4b63-94d7-d059f407c652 None None] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:31:22.444727 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2120a9-836d-4b63-94d7-d059f407c652 None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:31:22.445014 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2120a9-836d-4b63-94d7-d059f407c652 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:31:22.460550 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 833/1667] 10.222.0.49 () {60 vars in 1143 bytes} [Wed Jun 15 10:31:22 2022] POST /identity/v3/users => generated 292 bytes in 324 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 10:31:22.462370 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463cb227-242b-4510-9ccc-fb41187b8eaf admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:31:22.505190 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.auth.core [None req-ee2120a9-836d-4b63-94d7-d059f407c652 None None] MFA Rules not processed for user `6d152f74395c4b6cb889058f612d502f`. Rule list: `[]` (Enabled: `True`). {{(pid=80119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:31:22.508252 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463cb227-242b-4510-9ccc-fb41187b8eaf admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:31:22.510851 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-ee2120a9-836d-4b63-94d7-d059f407c652 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:31:22.519146 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5deb200-3b87-4891-a4b1-b51c1f4a3828 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:31:22.519742 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5deb200-3b87-4891-a4b1-b51c1f4a3828 admin admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:31:22.520033 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5deb200-3b87-4891-a4b1-b51c1f4a3828 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:31:22.523856 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5deb200-3b87-4891-a4b1-b51c1f4a3828 admin admin] PATH_INFO: `/v3/roles` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:31:22.526528 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5deb200-3b87-4891-a4b1-b51c1f4a3828 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:31:22.534445 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5deb200-3b87-4891-a4b1-b51c1f4a3828 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:31:22.552323 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 834/1668] 10.222.0.49 () {58 vars in 1121 bytes} [Wed Jun 15 10:31:22 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 10:31:22.556384 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5deb200-3b87-4891-a4b1-b51c1f4a3828 admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:31:22.591000 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5deb200-3b87-4891-a4b1-b51c1f4a3828 admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:31:22.604272 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c3ca74-bd71-4153-b80a-0bdb18a8520e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:31:22.605383 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c3ca74-bd71-4153-b80a-0bdb18a8520e admin admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:31:22.605882 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c3ca74-bd71-4153-b80a-0bdb18a8520e admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:31:22.606408 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c3ca74-bd71-4153-b80a-0bdb18a8520e admin admin] PATH_INFO: `/v3/roles` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:31:22.610267 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c3ca74-bd71-4153-b80a-0bdb18a8520e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:31:22.612966 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c3ca74-bd71-4153-b80a-0bdb18a8520e admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:31:22.642383 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 835/1669] 10.222.0.49 () {58 vars in 1121 bytes} [Wed Jun 15 10:31:22 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 15 10:31:22.648412 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c3ca74-bd71-4153-b80a-0bdb18a8520e admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:31:22.669902 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c3ca74-bd71-4153-b80a-0bdb18a8520e admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:31:22.671687 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 835/1670] 10.222.0.49 () {58 vars in 957 bytes} [Wed Jun 15 10:31:22 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 10:31:22.683451 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25294500-5ff6-4adc-9986-3c30367b4b2c None None] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:31:22.683751 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25294500-5ff6-4adc-9986-3c30367b4b2c None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:31:22.684034 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25294500-5ff6-4adc-9986-3c30367b4b2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:31:22.694959 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05f90fc-d647-4a5e-9851-b5e621fa6a81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:31:22.695875 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05f90fc-d647-4a5e-9851-b5e621fa6a81 admin admin] REQUEST_METHOD: `PUT` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:31:22.696159 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05f90fc-d647-4a5e-9851-b5e621fa6a81 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:31:22.696434 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05f90fc-d647-4a5e-9851-b5e621fa6a81 admin admin] PATH_INFO: `/v3/projects/086345a5a5b8414bb44888eb24463686/users/a6f02f41be834fe6b1ae803ae80ef6ee/roles/6876e57cc51c448097ab109d35b2a3da` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:31:22.712815 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.auth.core [None req-25294500-5ff6-4adc-9986-3c30367b4b2c None None] MFA Rules not processed for user `6d152f74395c4b6cb889058f612d502f`. Rule list: `[]` (Enabled: `True`). {{(pid=80119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:31:22.714196 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-25294500-5ff6-4adc-9986-3c30367b4b2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:31:22.734992 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 836/1671] 10.222.0.49 () {58 vars in 957 bytes} [Wed Jun 15 10:31:22 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:31:22.738532 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25294500-5ff6-4adc-9986-3c30367b4b2c None None] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:31:22.742678 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-25294500-5ff6-4adc-9986-3c30367b4b2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:31:22.753353 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05f90fc-d647-4a5e-9851-b5e621fa6a81 admin admin] RBAC: Authorizing `identity:create_grant(project_id=086345a5a5b8414bb44888eb24463686, user_id=a6f02f41be834fe6b1ae803ae80ef6ee, role_id=6876e57cc51c448097ab109d35b2a3da)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:31:22.756018 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05f90fc-d647-4a5e-9851-b5e621fa6a81 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:31:22.778915 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25294500-5ff6-4adc-9986-3c30367b4b2c None None] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:31:22.808135 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 836/1672] 10.222.0.49 () {60 vars in 1482 bytes} [Wed Jun 15 10:31:22 2022] PUT /identity/v3/projects/086345a5a5b8414bb44888eb24463686/users/a6f02f41be834fe6b1ae803ae80ef6ee/roles/6876e57cc51c448097ab109d35b2a3da => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:31:22.809230 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98c3d4c-8dae-432d-8453-d4f5e5aa8e9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:31:22.809353 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05f90fc-d647-4a5e-9851-b5e621fa6a81 admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:31:22.810713 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98c3d4c-8dae-432d-8453-d4f5e5aa8e9c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:31:22.811304 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-f05f90fc-d647-4a5e-9851-b5e621fa6a81 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:31:22.811872 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98c3d4c-8dae-432d-8453-d4f5e5aa8e9c admin admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:31:22.812944 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98c3d4c-8dae-432d-8453-d4f5e5aa8e9c admin admin] PATH_INFO: `/v3/projects/44fccf2eb3e14011b997d90302f4e59f` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:31:22.816094 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f98c3d4c-8dae-432d-8453-d4f5e5aa8e9c admin admin] RBAC: Authorizing `identity:delete_project(project_id=44fccf2eb3e14011b997d90302f4e59f)` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:31:22.827592 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f98c3d4c-8dae-432d-8453-d4f5e5aa8e9c admin admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:31:22.922089 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:31:23.026218 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-f98c3d4c-8dae-432d-8453-d4f5e5aa8e9c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '44fccf2eb3e14011b997d90302f4e59f', 'request_id': 'req-f98c3d4c-8dae-432d-8453-d4f5e5aa8e9c'} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:31:23.043987 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-f98c3d4c-8dae-432d-8453-d4f5e5aa8e9c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '44fccf2eb3e14011b997d90302f4e59f', 'request_id': 'req-f98c3d4c-8dae-432d-8453-d4f5e5aa8e9c'} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:31:23.052055 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:DELETE=13,keystone:SELECT=84,keystone:INSERT=3 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:31:23.054260 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-f98c3d4c-8dae-432d-8453-d4f5e5aa8e9c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '44fccf2eb3e14011b997d90302f4e59f', 'request_id': 'req-f98c3d4c-8dae-432d-8453-d4f5e5aa8e9c'} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:31:23.056640 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05f90fc-d647-4a5e-9851-b5e621fa6a81 admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:31:23.079601 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-f98c3d4c-8dae-432d-8453-d4f5e5aa8e9c admin admin] The token cache is being invalidate because project 44fccf2eb3e14011b997d90302f4e59f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80119) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 10:31:23.081362 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-f98c3d4c-8dae-432d-8453-d4f5e5aa8e9c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:31:23.082168 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-f98c3d4c-8dae-432d-8453-d4f5e5aa8e9c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:31:23.088690 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 837/1673] 10.222.0.49 () {58 vars in 1232 bytes} [Wed Jun 15 10:31:22 2022] DELETE /identity/v3/projects/44fccf2eb3e14011b997d90302f4e59f => generated 0 bytes in 353 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:31:23.089953 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49ee57b-be0e-4edf-ae3d-36709f99495d None None] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:31:23.090255 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49ee57b-be0e-4edf-ae3d-36709f99495d None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:31:23.090535 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49ee57b-be0e-4edf-ae3d-36709f99495d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:31:23.153935 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.auth.core [None req-c49ee57b-be0e-4edf-ae3d-36709f99495d None None] MFA Rules not processed for user `a6f02f41be834fe6b1ae803ae80ef6ee`. Rule list: `[]` (Enabled: `True`). {{(pid=80119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:31:23.155302 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-c49ee57b-be0e-4edf-ae3d-36709f99495d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:31:23.208912 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6eea99-3eb1-4bb9-b575-f881c0ed662e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ac54528370d4747b6e8777fba9d0c82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '559b3e713f5844b6ae37151d56dd419f', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:31:23.209682 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6eea99-3eb1-4bb9-b575-f881c0ed662e service nova] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:31:23.210042 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6eea99-3eb1-4bb9-b575-f881c0ed662e service nova] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:31:23.210394 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6eea99-3eb1-4bb9-b575-f881c0ed662e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:31:23.212433 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-6a6eea99-3eb1-4bb9-b575-f881c0ed662e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:31:23.291602 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a6eea99-3eb1-4bb9-b575-f881c0ed662e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:31:23.294007 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a6eea99-3eb1-4bb9-b575-f881c0ed662e service nova] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:31:23.320492 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 837/1674] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Jun 15 10:31:22 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 512 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:31:23.321563 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6eea99-3eb1-4bb9-b575-f881c0ed662e service nova] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:31:23.322760 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 838/1675] 10.222.0.49 () {58 vars in 957 bytes} [Wed Jun 15 10:31:23 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 234 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:31:23.327134 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-6a6eea99-3eb1-4bb9-b575-f881c0ed662e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:31:23.327283 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49ee57b-be0e-4edf-ae3d-36709f99495d None None] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:31:23.329112 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-c49ee57b-be0e-4edf-ae3d-36709f99495d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:31:23.515040 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49ee57b-be0e-4edf-ae3d-36709f99495d None None] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:31:23.521614 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6eea99-3eb1-4bb9-b575-f881c0ed662e service nova] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:31:23.537226 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a711d096-260d-4ada-8eac-b92bb851d413 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:31:23.537915 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a711d096-260d-4ada-8eac-b92bb851d413 admin admin] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:31:23.538203 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a711d096-260d-4ada-8eac-b92bb851d413 admin admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:31:23.538482 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a711d096-260d-4ada-8eac-b92bb851d413 admin admin] PATH_INFO: `/v3/projects` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:31:23.539233 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a711d096-260d-4ada-8eac-b92bb851d413 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:31:23.541207 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a711d096-260d-4ada-8eac-b92bb851d413 admin admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:31:23.544829 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4149640-507d-4a82-b381-f26801da6d69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:31:23.545735 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4149640-507d-4a82-b381-f26801da6d69 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:31:23.546017 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4149640-507d-4a82-b381-f26801da6d69 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:31:23.546293 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4149640-507d-4a82-b381-f26801da6d69 admin admin] PATH_INFO: `/v3/users/7c71258089f34535b5dbb17a781b3115` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:31:23.547602 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4149640-507d-4a82-b381-f26801da6d69 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7c71258089f34535b5dbb17a781b3115)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:31:23.549607 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4149640-507d-4a82-b381-f26801da6d69 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:31:23.579452 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:31:23.604289 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 839/1676] 10.222.0.49 () {60 vars in 1152 bytes} [Wed Jun 15 10:31:23 2022] POST /identity/v3/projects => generated 383 bytes in 278 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 10:31:23.605261 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a711d096-260d-4ada-8eac-b92bb851d413 admin admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:31:23.654144 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-c4149640-507d-4a82-b381-f26801da6d69 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7c71258089f34535b5dbb17a781b3115'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:31:23.668202 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-c4149640-507d-4a82-b381-f26801da6d69 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7c71258089f34535b5dbb17a781b3115'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:31:23.669025 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-c4149640-507d-4a82-b381-f26801da6d69 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7c71258089f34535b5dbb17a781b3115'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:31:23.669676 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-c4149640-507d-4a82-b381-f26801da6d69 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7c71258089f34535b5dbb17a781b3115'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:31:23.679860 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-c4149640-507d-4a82-b381-f26801da6d69 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7c71258089f34535b5dbb17a781b3115'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:31:23.712049 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 838/1677] 10.222.0.49 () {58 vars in 1223 bytes} [Wed Jun 15 10:31:23 2022] DELETE /identity/v3/users/7c71258089f34535b5dbb17a781b3115 => generated 0 bytes in 390 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:31:23.782695 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a711d096-260d-4ada-8eac-b92bb851d413 admin admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:31:23.802252 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:31:23.954364 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8daea006-9474-49d6-adb7-3718fa8d6a40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:31:23.955076 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8daea006-9474-49d6-adb7-3718fa8d6a40 admin admin] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:31:23.955383 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8daea006-9474-49d6-adb7-3718fa8d6a40 admin admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:31:23.955661 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8daea006-9474-49d6-adb7-3718fa8d6a40 admin admin] PATH_INFO: `/v3/users` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:31:23.956363 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8daea006-9474-49d6-adb7-3718fa8d6a40 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:31:23.958328 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8daea006-9474-49d6-adb7-3718fa8d6a40 admin admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:31:24.017648 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 840/1678] 10.222.0.49 () {60 vars in 1143 bytes} [Wed Jun 15 10:31:23 2022] POST /identity/v3/users => generated 298 bytes in 410 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 10:31:24.018846 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4149640-507d-4a82-b381-f26801da6d69 admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:31:24.020324 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-c4149640-507d-4a82-b381-f26801da6d69 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:31:24.045913 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4149640-507d-4a82-b381-f26801da6d69 admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:31:24.063012 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db1425b-0c4a-46b0-834e-ea2751a51476 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:31:24.067151 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db1425b-0c4a-46b0-834e-ea2751a51476 admin admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:31:24.067616 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db1425b-0c4a-46b0-834e-ea2751a51476 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:31:24.068036 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db1425b-0c4a-46b0-834e-ea2751a51476 admin admin] PATH_INFO: `/v3/roles` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:31:24.068927 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db1425b-0c4a-46b0-834e-ea2751a51476 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:31:24.070726 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db1425b-0c4a-46b0-834e-ea2751a51476 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:31:24.087355 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 839/1679] 10.222.0.49 () {58 vars in 1121 bytes} [Wed Jun 15 10:31:24 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 10:31:24.091162 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8daea006-9474-49d6-adb7-3718fa8d6a40 admin admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:31:24.106684 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8daea006-9474-49d6-adb7-3718fa8d6a40 admin admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:31:24.139315 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726aa901-cc34-4b34-93a5-6b00d5f67906 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:31:24.140351 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726aa901-cc34-4b34-93a5-6b00d5f67906 admin admin] REQUEST_METHOD: `PUT` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:31:24.140463 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726aa901-cc34-4b34-93a5-6b00d5f67906 admin admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:31:24.140700 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726aa901-cc34-4b34-93a5-6b00d5f67906 admin admin] PATH_INFO: `/v3/projects/d56733aa8e8342e3b70961a9476f2005/users/ebd5a5c7cd594a69aec95ba00d4f5612/roles/8051174e7a7947d5b991ebd793c2e543` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:31:24.187240 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-726aa901-cc34-4b34-93a5-6b00d5f67906 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d56733aa8e8342e3b70961a9476f2005, user_id=ebd5a5c7cd594a69aec95ba00d4f5612, role_id=8051174e7a7947d5b991ebd793c2e543)` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:31:24.190088 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-726aa901-cc34-4b34-93a5-6b00d5f67906 admin admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:31:24.232398 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 841/1680] 10.222.0.49 () {60 vars in 1482 bytes} [Wed Jun 15 10:31:24 2022] PUT /identity/v3/projects/d56733aa8e8342e3b70961a9476f2005/users/ebd5a5c7cd594a69aec95ba00d4f5612/roles/8051174e7a7947d5b991ebd793c2e543 => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:31:24.237225 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db1425b-0c4a-46b0-834e-ea2751a51476 admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:31:24.250560 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db1425b-0c4a-46b0-834e-ea2751a51476 admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:31:24.385708 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d434a4b8-1842-41c5-844f-a2808407fc32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:31:24.386429 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d434a4b8-1842-41c5-844f-a2808407fc32 admin admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:31:24.386787 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d434a4b8-1842-41c5-844f-a2808407fc32 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:31:24.387164 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d434a4b8-1842-41c5-844f-a2808407fc32 admin admin] PATH_INFO: `/v3/roles` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:31:24.388511 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d434a4b8-1842-41c5-844f-a2808407fc32 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:31:24.391302 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d434a4b8-1842-41c5-844f-a2808407fc32 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:31:24.407564 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 840/1681] 10.222.0.49 () {58 vars in 1121 bytes} [Wed Jun 15 10:31:24 2022] GET /identity/v3/roles => generated 1420 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 10:31:24.407799 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726aa901-cc34-4b34-93a5-6b00d5f67906 admin admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:31:24.422987 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726aa901-cc34-4b34-93a5-6b00d5f67906 admin admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:31:24.436824 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0d450f-66cb-4522-906c-537fd7078188 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:31:24.437749 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0d450f-66cb-4522-906c-537fd7078188 admin admin] REQUEST_METHOD: `PUT` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:31:24.438040 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0d450f-66cb-4522-906c-537fd7078188 admin admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:31:24.438318 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0d450f-66cb-4522-906c-537fd7078188 admin admin] PATH_INFO: `/v3/projects/d56733aa8e8342e3b70961a9476f2005/users/ebd5a5c7cd594a69aec95ba00d4f5612/roles/8051174e7a7947d5b991ebd793c2e543` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:31:24.443846 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0d450f-66cb-4522-906c-537fd7078188 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d56733aa8e8342e3b70961a9476f2005, user_id=ebd5a5c7cd594a69aec95ba00d4f5612, role_id=8051174e7a7947d5b991ebd793c2e543)` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:31:24.446548 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0d450f-66cb-4522-906c-537fd7078188 admin admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:31:24.464701 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d2e300-f0ac-4cf0-b668-04e38df22cf4 None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:31:24.465229 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d2e300-f0ac-4cf0-b668-04e38df22cf4 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:31:24.467476 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d2e300-f0ac-4cf0-b668-04e38df22cf4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:31:24.475482 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 842/1682] 10.222.0.49 () {60 vars in 1482 bytes} [Wed Jun 15 10:31:24 2022] PUT /identity/v3/projects/d56733aa8e8342e3b70961a9476f2005/users/ebd5a5c7cd594a69aec95ba00d4f5612/roles/8051174e7a7947d5b991ebd793c2e543 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:31:24.487032 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc04d5e-298a-4653-bc06-aee5d55db9c1 None None] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:31:24.489755 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc04d5e-298a-4653-bc06-aee5d55db9c1 None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:31:24.490046 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc04d5e-298a-4653-bc06-aee5d55db9c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:31:24.521210 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-e9d2e300-f0ac-4cf0-b668-04e38df22cf4 None None] MFA Rules not processed for user `103e10244eca444b86b9c51ff314d445`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:31:24.522379 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-e9d2e300-f0ac-4cf0-b668-04e38df22cf4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:31:24.568779 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.auth.core [None req-bfc04d5e-298a-4653-bc06-aee5d55db9c1 None None] MFA Rules not processed for user `ebd5a5c7cd594a69aec95ba00d4f5612`. Rule list: `[]` (Enabled: `True`). {{(pid=80119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:31:24.599790 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-bfc04d5e-298a-4653-bc06-aee5d55db9c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:31:24.658037 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 841/1683] 10.222.0.49 () {58 vars in 1009 bytes} [Wed Jun 15 10:31:24 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 10:31:24.662090 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0222790-f543-4e52-a7e3-54c5be9688aa None None] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:31:24.666445 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0222790-f543-4e52-a7e3-54c5be9688aa None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:31:24.666968 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0222790-f543-4e52-a7e3-54c5be9688aa None None] PATH_INFO: `/` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:31:24.668772 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 842/1684] 10.222.0.49 () {54 vars in 912 bytes} [Wed Jun 15 10:31:24 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:31:24.674573 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32673ffc-3412-46b3-8c4c-c86391be7fb5 None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:31:24.674971 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32673ffc-3412-46b3-8c4c-c86391be7fb5 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:31:24.675365 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32673ffc-3412-46b3-8c4c-c86391be7fb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:31:24.729325 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-32673ffc-3412-46b3-8c4c-c86391be7fb5 None None] MFA Rules not processed for user `0cb44a17d205431793e9a8791e477a32`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:31:24.732475 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-32673ffc-3412-46b3-8c4c-c86391be7fb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:31:24.804571 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:DELETE=9,keystone:SELECT=90,keystone:INSERT=1 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:31:24.826667 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 843/1685] 10.222.0.49 () {58 vars in 957 bytes} [Wed Jun 15 10:31:24 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 341 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:31:24.843789 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc04d5e-298a-4653-bc06-aee5d55db9c1 None None] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:31:24.848973 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-bfc04d5e-298a-4653-bc06-aee5d55db9c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:31:24.862175 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 843/1686] 10.222.0.49 () {58 vars in 1009 bytes} [Wed Jun 15 10:31:24 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:31:24.945978 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32673ffc-3412-46b3-8c4c-c86391be7fb5 None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:31:24.949518 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-32673ffc-3412-46b3-8c4c-c86391be7fb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:31:24.965483 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc04d5e-298a-4653-bc06-aee5d55db9c1 None None] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:31:24.971027 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd2d2c8-d52c-4cbd-b9da-904ce8629dc5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20d1327553404b80aecee87c78fcb52b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '559b3e713f5844b6ae37151d56dd419f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:31:24.972007 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd2d2c8-d52c-4cbd-b9da-904ce8629dc5 service cinder] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:31:24.972494 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd2d2c8-d52c-4cbd-b9da-904ce8629dc5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:31:24.973001 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd2d2c8-d52c-4cbd-b9da-904ce8629dc5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:31:24.982384 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:31:24.985573 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd2d2c8-d52c-4cbd-b9da-904ce8629dc5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:31:24.987523 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd2d2c8-d52c-4cbd-b9da-904ce8629dc5 service cinder] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:31:25.005260 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 844/1687] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Jun 15 10:31:24 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:31:25.109001 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32673ffc-3412-46b3-8c4c-c86391be7fb5 None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:31:25.113946 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e60795-a956-49d8-a62f-c21de487d29c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a69f99210444f9696e8b22615fb4a41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '559b3e713f5844b6ae37151d56dd419f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:31:25.114752 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e60795-a956-49d8-a62f-c21de487d29c service swift] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:31:25.116546 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e60795-a956-49d8-a62f-c21de487d29c service swift] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:31:25.116825 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e60795-a956-49d8-a62f-c21de487d29c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:31:25.136589 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e60795-a956-49d8-a62f-c21de487d29c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:31:25.156980 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e60795-a956-49d8-a62f-c21de487d29c service swift] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:31:25.187029 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 844/1688] 10.222.0.49 () {60 vars in 1384 bytes} [Wed Jun 15 10:31:24 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 241 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jun 15 10:31:25.281965 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd2d2c8-d52c-4cbd-b9da-904ce8629dc5 service cinder] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:31:25.283976 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-ddd2d2c8-d52c-4cbd-b9da-904ce8629dc5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:31:25.301307 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd2d2c8-d52c-4cbd-b9da-904ce8629dc5 service cinder] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:31:25.316404 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5f6cdc-aae9-4fb8-8b97-f7e9099ccc05 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20d1327553404b80aecee87c78fcb52b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '559b3e713f5844b6ae37151d56dd419f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:31:25.317613 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5f6cdc-aae9-4fb8-8b97-f7e9099ccc05 service cinder] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:31:25.317613 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5f6cdc-aae9-4fb8-8b97-f7e9099ccc05 service cinder] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:31:25.317958 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5f6cdc-aae9-4fb8-8b97-f7e9099ccc05 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:31:25.324381 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-5b5f6cdc-aae9-4fb8-8b97-f7e9099ccc05 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:31:25.504794 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5f6cdc-aae9-4fb8-8b97-f7e9099ccc05 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:31:25.506417 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5f6cdc-aae9-4fb8-8b97-f7e9099ccc05 service cinder] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:31:25.546731 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 845/1689] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Jun 15 10:31:25 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 265 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:31:25.762119 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e60795-a956-49d8-a62f-c21de487d29c service swift] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:31:25.763693 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-21e60795-a956-49d8-a62f-c21de487d29c service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:31:26.001580 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5f6cdc-aae9-4fb8-8b97-f7e9099ccc05 service cinder] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:31:26.003394 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-5b5f6cdc-aae9-4fb8-8b97-f7e9099ccc05 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:31:26.016877 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e60795-a956-49d8-a62f-c21de487d29c service swift] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:31:26.037017 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5f6cdc-aae9-4fb8-8b97-f7e9099ccc05 service cinder] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:31:26.040185 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c7a518-cb65-477d-a8f8-2b82b49c05cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:31:26.041544 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c7a518-cb65-477d-a8f8-2b82b49c05cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:31:26.042040 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c7a518-cb65-477d-a8f8-2b82b49c05cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:31:26.042943 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c7a518-cb65-477d-a8f8-2b82b49c05cf admin admin] PATH_INFO: `/v3/projects/888edbc1e4ba47debf36ca4f85b4f446` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:31:26.045563 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c7a518-cb65-477d-a8f8-2b82b49c05cf admin admin] RBAC: Authorizing `identity:delete_project(project_id=888edbc1e4ba47debf36ca4f85b4f446)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:31:26.065611 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: WARNING py.warnings [None req-32c7a518-cb65-477d-a8f8-2b82b49c05cf admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 15 10:31:26.065611 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: warnings.warn(msg) Jun 15 10:31:26.065611 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: Jun 15 10:31:26.071235 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e3542f-bb9c-4298-a7c0-943dfd2c3928 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0cb44a17d205431793e9a8791e477a32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '559b3e713f5844b6ae37151d56dd419f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:31:26.073519 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e3542f-bb9c-4298-a7c0-943dfd2c3928 service neutron] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:31:26.077746 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e3542f-bb9c-4298-a7c0-943dfd2c3928 service neutron] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:31:26.078755 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c7a518-cb65-477d-a8f8-2b82b49c05cf admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:31:26.079788 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e3542f-bb9c-4298-a7c0-943dfd2c3928 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:31:26.103578 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e3542f-bb9c-4298-a7c0-943dfd2c3928 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:31:26.117858 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e3542f-bb9c-4298-a7c0-943dfd2c3928 service neutron] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:31:26.150471 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 846/1690] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Jun 15 10:31:25 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 10:31:26.332956 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-32c7a518-cb65-477d-a8f8-2b82b49c05cf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '888edbc1e4ba47debf36ca4f85b4f446', 'request_id': 'req-32c7a518-cb65-477d-a8f8-2b82b49c05cf'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:31:26.349516 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-32c7a518-cb65-477d-a8f8-2b82b49c05cf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '888edbc1e4ba47debf36ca4f85b4f446', 'request_id': 'req-32c7a518-cb65-477d-a8f8-2b82b49c05cf'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:31:26.366227 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-32c7a518-cb65-477d-a8f8-2b82b49c05cf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '888edbc1e4ba47debf36ca4f85b4f446', 'request_id': 'req-32c7a518-cb65-477d-a8f8-2b82b49c05cf'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:31:26.371655 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:31:26.440545 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-32c7a518-cb65-477d-a8f8-2b82b49c05cf admin admin] The token cache is being invalidate because project 888edbc1e4ba47debf36ca4f85b4f446 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80120) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 10:31:26.441585 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-32c7a518-cb65-477d-a8f8-2b82b49c05cf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:31:26.446119 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-32c7a518-cb65-477d-a8f8-2b82b49c05cf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:31:26.461362 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 845/1691] 10.222.0.49 () {58 vars in 1232 bytes} [Wed Jun 15 10:31:25 2022] DELETE /identity/v3/projects/888edbc1e4ba47debf36ca4f85b4f446 => generated 0 bytes in 693 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:31:26.461461 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448ff438-0094-4658-a655-a47e7bd928eb None None] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:31:26.461461 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448ff438-0094-4658-a655-a47e7bd928eb None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:31:26.461461 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448ff438-0094-4658-a655-a47e7bd928eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:31:26.478789 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c7a518-cb65-477d-a8f8-2b82b49c05cf admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:31:26.482582 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-32c7a518-cb65-477d-a8f8-2b82b49c05cf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:31:26.535203 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.auth.core [None req-448ff438-0094-4658-a655-a47e7bd928eb None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:31:26.543602 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-448ff438-0094-4658-a655-a47e7bd928eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:31:26.763572 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c7a518-cb65-477d-a8f8-2b82b49c05cf admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:31:26.777915 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ae69d8-90d3-43e4-8876-1460f2edffeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:31:26.779067 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ae69d8-90d3-43e4-8876-1460f2edffeb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:31:26.779708 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ae69d8-90d3-43e4-8876-1460f2edffeb admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:31:26.779973 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ae69d8-90d3-43e4-8876-1460f2edffeb admin admin] PATH_INFO: `/v3/users/a0652c42307246b49ee7728aab8c4ba9` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:31:26.783791 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ae69d8-90d3-43e4-8876-1460f2edffeb admin admin] RBAC: Authorizing `identity:delete_user(user_id=a0652c42307246b49ee7728aab8c4ba9)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:31:26.785941 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ae69d8-90d3-43e4-8876-1460f2edffeb admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:31:26.823328 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 847/1692] 10.222.0.49 () {58 vars in 957 bytes} [Wed Jun 15 10:31:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 362 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:31:26.829376 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6eb0806-070f-4f54-aee2-18f9476ec894 None None] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:31:26.829783 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6eb0806-070f-4f54-aee2-18f9476ec894 None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:31:26.830821 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6eb0806-070f-4f54-aee2-18f9476ec894 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:31:26.896104 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-88ae69d8-90d3-43e4-8876-1460f2edffeb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a0652c42307246b49ee7728aab8c4ba9'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:31:26.897125 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.auth.core [None req-d6eb0806-070f-4f54-aee2-18f9476ec894 None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:31:26.903191 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-d6eb0806-070f-4f54-aee2-18f9476ec894 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:31:26.912202 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-88ae69d8-90d3-43e4-8876-1460f2edffeb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a0652c42307246b49ee7728aab8c4ba9'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:31:26.913947 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-88ae69d8-90d3-43e4-8876-1460f2edffeb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a0652c42307246b49ee7728aab8c4ba9'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:31:26.916158 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-88ae69d8-90d3-43e4-8876-1460f2edffeb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a0652c42307246b49ee7728aab8c4ba9'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:31:26.930179 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-88ae69d8-90d3-43e4-8876-1460f2edffeb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a0652c42307246b49ee7728aab8c4ba9'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:31:26.934991 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 848/1693] 10.222.0.49 () {58 vars in 957 bytes} [Wed Jun 15 10:31:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:31:26.940725 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6eb0806-070f-4f54-aee2-18f9476ec894 None None] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:31:26.975639 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6eb0806-070f-4f54-aee2-18f9476ec894 None None] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:31:26.990771 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4936d23-c3b2-4574-a41c-5e328475b26d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:31:26.992508 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4936d23-c3b2-4574-a41c-5e328475b26d admin admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:31:26.994340 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4936d23-c3b2-4574-a41c-5e328475b26d admin admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:31:26.994776 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4936d23-c3b2-4574-a41c-5e328475b26d admin admin] PATH_INFO: `/v3/domains` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:31:26.998413 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4936d23-c3b2-4574-a41c-5e328475b26d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:31:26.999018 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4936d23-c3b2-4574-a41c-5e328475b26d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:31:27.003557 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4936d23-c3b2-4574-a41c-5e328475b26d admin admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:31:27.013620 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 846/1694] 10.222.0.49 () {58 vars in 1223 bytes} [Wed Jun 15 10:31:26 2022] DELETE /identity/v3/users/a0652c42307246b49ee7728aab8c4ba9 => generated 0 bytes in 531 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:31:27.015387 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d0c628-063a-4d76-a2e3-5c02d31e91b1 None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:31:27.015678 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d0c628-063a-4d76-a2e3-5c02d31e91b1 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:31:27.015967 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d0c628-063a-4d76-a2e3-5c02d31e91b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:31:27.048427 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 849/1695] 10.222.0.49 () {58 vars in 1152 bytes} [Wed Jun 15 10:31:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:31:27.055509 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4936d23-c3b2-4574-a41c-5e328475b26d admin admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:31:27.092312 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4936d23-c3b2-4574-a41c-5e328475b26d admin admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:31:27.121058 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-c1d0c628-063a-4d76-a2e3-5c02d31e91b1 None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:31:27.131110 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-c1d0c628-063a-4d76-a2e3-5c02d31e91b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:31:27.279582 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81619fca-1846-4211-8056-a98fa9bc3995 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:31:27.280398 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81619fca-1846-4211-8056-a98fa9bc3995 admin admin] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:31:27.280917 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81619fca-1846-4211-8056-a98fa9bc3995 admin admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:31:27.281297 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81619fca-1846-4211-8056-a98fa9bc3995 admin admin] PATH_INFO: `/v3/projects` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:31:27.282103 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81619fca-1846-4211-8056-a98fa9bc3995 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:31:27.284991 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81619fca-1846-4211-8056-a98fa9bc3995 admin admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:31:27.293769 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=100 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:31:27.343154 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 850/1696] 10.222.0.49 () {60 vars in 1152 bytes} [Wed Jun 15 10:31:27 2022] POST /identity/v3/projects => generated 389 bytes in 290 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 10:31:27.347448 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81619fca-1846-4211-8056-a98fa9bc3995 admin admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:31:27.380462 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 847/1697] 10.222.0.49 () {58 vars in 957 bytes} [Wed Jun 15 10:31:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 366 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:31:27.413737 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d0c628-063a-4d76-a2e3-5c02d31e91b1 None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:31:27.415898 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-c1d0c628-063a-4d76-a2e3-5c02d31e91b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:31:27.445341 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:31:27.506209 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81619fca-1846-4211-8056-a98fa9bc3995 admin admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:31:27.520159 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b00eef8-2bd2-4197-a008-ab841d25a9bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:31:27.521016 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b00eef8-2bd2-4197-a008-ab841d25a9bd admin admin] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:31:27.521402 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b00eef8-2bd2-4197-a008-ab841d25a9bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:31:27.521805 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b00eef8-2bd2-4197-a008-ab841d25a9bd admin admin] PATH_INFO: `/v3/users` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:31:27.522778 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b00eef8-2bd2-4197-a008-ab841d25a9bd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:31:27.528352 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b00eef8-2bd2-4197-a008-ab841d25a9bd admin admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:31:27.570870 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 851/1698] 10.222.0.49 () {60 vars in 1143 bytes} [Wed Jun 15 10:31:27 2022] POST /identity/v3/users => generated 295 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 10:31:27.574258 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b00eef8-2bd2-4197-a008-ab841d25a9bd admin admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:31:27.596024 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d0c628-063a-4d76-a2e3-5c02d31e91b1 None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:31:27.630876 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b00eef8-2bd2-4197-a008-ab841d25a9bd admin admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:31:27.662410 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9836eb6-0371-431e-8c95-749ab66c760c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:31:27.663018 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9836eb6-0371-431e-8c95-749ab66c760c admin admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:31:27.663746 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9836eb6-0371-431e-8c95-749ab66c760c admin admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:31:27.663746 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9836eb6-0371-431e-8c95-749ab66c760c admin admin] PATH_INFO: `/v3/roles` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:31:27.664355 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9836eb6-0371-431e-8c95-749ab66c760c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:31:27.672880 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9836eb6-0371-431e-8c95-749ab66c760c admin admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:31:27.733435 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b17d87e-86cc-4cc0-8058-34509bd6c24e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0cb44a17d205431793e9a8791e477a32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '559b3e713f5844b6ae37151d56dd419f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:31:27.734934 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b17d87e-86cc-4cc0-8058-34509bd6c24e service neutron] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:31:27.736020 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b17d87e-86cc-4cc0-8058-34509bd6c24e service neutron] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:31:27.736375 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b17d87e-86cc-4cc0-8058-34509bd6c24e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:31:27.751498 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b17d87e-86cc-4cc0-8058-34509bd6c24e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:31:27.755417 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 852/1699] 10.222.0.49 () {58 vars in 1121 bytes} [Wed Jun 15 10:31:27 2022] GET /identity/v3/roles => generated 1420 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 10:31:27.755575 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b17d87e-86cc-4cc0-8058-34509bd6c24e service neutron] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:31:27.768176 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9836eb6-0371-431e-8c95-749ab66c760c admin admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:31:27.790462 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 848/1700] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Jun 15 10:31:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 10:31:27.828756 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9836eb6-0371-431e-8c95-749ab66c760c admin admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:31:27.858254 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92bffa37-985a-4de3-8658-41908e95d80b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:31:27.861824 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bffa37-985a-4de3-8658-41908e95d80b admin admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:31:27.870430 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bffa37-985a-4de3-8658-41908e95d80b admin admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:31:27.870901 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bffa37-985a-4de3-8658-41908e95d80b admin admin] PATH_INFO: `/v3/roles` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:31:27.876608 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92bffa37-985a-4de3-8658-41908e95d80b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:31:27.878074 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92bffa37-985a-4de3-8658-41908e95d80b admin admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:31:27.943058 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 853/1701] 10.222.0.49 () {58 vars in 1121 bytes} [Wed Jun 15 10:31:27 2022] GET /identity/v3/roles => generated 1420 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 10:31:27.944140 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b17d87e-86cc-4cc0-8058-34509bd6c24e service neutron] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:31:27.961418 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b17d87e-86cc-4cc0-8058-34509bd6c24e service neutron] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:31:27.987443 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5ef64e-ab1b-4b82-af39-3786b066a712 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:31:27.988406 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5ef64e-ab1b-4b82-af39-3786b066a712 admin admin] REQUEST_METHOD: `PUT` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:31:27.988691 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5ef64e-ab1b-4b82-af39-3786b066a712 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:31:27.988968 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5ef64e-ab1b-4b82-af39-3786b066a712 admin admin] PATH_INFO: `/v3/projects/ef878af2a9184c309f244b3be501fd73/users/1f73691a6eac4f009faa844945a797dd/roles/6876e57cc51c448097ab109d35b2a3da` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:31:28.034396 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874064b6-2f41-4110-9ef4-b9a1117449ff None None] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:31:28.034681 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874064b6-2f41-4110-9ef4-b9a1117449ff None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:31:28.035016 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874064b6-2f41-4110-9ef4-b9a1117449ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:31:28.085745 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.auth.core [None req-874064b6-2f41-4110-9ef4-b9a1117449ff None None] MFA Rules not processed for user `6d152f74395c4b6cb889058f612d502f`. Rule list: `[]` (Enabled: `True`). {{(pid=80119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:31:28.093143 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-874064b6-2f41-4110-9ef4-b9a1117449ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:31:28.129619 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a5ef64e-ab1b-4b82-af39-3786b066a712 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ef878af2a9184c309f244b3be501fd73, user_id=1f73691a6eac4f009faa844945a797dd, role_id=6876e57cc51c448097ab109d35b2a3da)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:31:28.170592 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a5ef64e-ab1b-4b82-af39-3786b066a712 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:31:28.260082 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 849/1702] 10.222.0.49 () {60 vars in 1482 bytes} [Wed Jun 15 10:31:27 2022] PUT /identity/v3/projects/ef878af2a9184c309f244b3be501fd73/users/1f73691a6eac4f009faa844945a797dd/roles/6876e57cc51c448097ab109d35b2a3da => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 10:31:28.271803 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a0de4c-4a49-47d7-9ff4-a9ea64607a48 None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:31:28.272102 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a0de4c-4a49-47d7-9ff4-a9ea64607a48 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:31:28.272381 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a0de4c-4a49-47d7-9ff4-a9ea64607a48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:31:28.358242 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 854/1703] 10.222.0.49 () {58 vars in 957 bytes} [Wed Jun 15 10:31:28 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 325 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:31:28.375514 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521ee84e-38db-43f4-aa20-df1c2cf4cf0e None None] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:31:28.375789 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521ee84e-38db-43f4-aa20-df1c2cf4cf0e None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:31:28.376023 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521ee84e-38db-43f4-aa20-df1c2cf4cf0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:31:28.396878 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-79a0de4c-4a49-47d7-9ff4-a9ea64607a48 None None] MFA Rules not processed for user `1f73691a6eac4f009faa844945a797dd`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:31:28.399214 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-79a0de4c-4a49-47d7-9ff4-a9ea64607a48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:31:28.479931 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.auth.core [None req-521ee84e-38db-43f4-aa20-df1c2cf4cf0e None None] MFA Rules not processed for user `0cb44a17d205431793e9a8791e477a32`. Rule list: `[]` (Enabled: `True`). {{(pid=80119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:31:28.481659 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-521ee84e-38db-43f4-aa20-df1c2cf4cf0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:31:28.673898 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 855/1704] 10.222.0.62 () {58 vars in 1015 bytes} [Wed Jun 15 10:31:28 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 306 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:31:28.720490 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521ee84e-38db-43f4-aa20-df1c2cf4cf0e None None] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:31:28.735790 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521ee84e-38db-43f4-aa20-df1c2cf4cf0e None None] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:31:28.747000 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7484138-189c-4564-9f3a-968b36d94002 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0cb44a17d205431793e9a8791e477a32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '559b3e713f5844b6ae37151d56dd419f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:31:28.752972 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7484138-189c-4564-9f3a-968b36d94002 service neutron] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:31:28.753735 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7484138-189c-4564-9f3a-968b36d94002 service neutron] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:31:28.754870 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7484138-189c-4564-9f3a-968b36d94002 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:31:28.760674 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 850/1705] 10.222.0.49 () {58 vars in 957 bytes} [Wed Jun 15 10:31:28 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 490 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 10:31:28.762919 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a0de4c-4a49-47d7-9ff4-a9ea64607a48 None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:31:28.775937 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:31:28.796674 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7484138-189c-4564-9f3a-968b36d94002 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:31:28.798607 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7484138-189c-4564-9f3a-968b36d94002 service neutron] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:31:28.825151 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 856/1706] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Jun 15 10:31:28 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 10:31:28.829070 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7484138-189c-4564-9f3a-968b36d94002 service neutron] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:31:28.831342 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-d7484138-189c-4564-9f3a-968b36d94002 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:31:28.983482 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:31:29.083791 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a0de4c-4a49-47d7-9ff4-a9ea64607a48 None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:31:29.108450 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea34754-8151-4b0c-b9d3-9f01f36f63ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:31:29.109200 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea34754-8151-4b0c-b9d3-9f01f36f63ae admin admin] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:31:29.110943 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea34754-8151-4b0c-b9d3-9f01f36f63ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:31:29.112752 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea34754-8151-4b0c-b9d3-9f01f36f63ae admin admin] PATH_INFO: `/v3/projects` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:31:29.113336 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea34754-8151-4b0c-b9d3-9f01f36f63ae admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:31:29.117805 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea34754-8151-4b0c-b9d3-9f01f36f63ae admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:31:29.152361 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7484138-189c-4564-9f3a-968b36d94002 service neutron] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:31:29.169333 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 851/1707] 10.222.0.49 () {60 vars in 1152 bytes} [Wed Jun 15 10:31:28 2022] POST /identity/v3/projects => generated 387 bytes in 408 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 10:31:29.174974 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea34754-8151-4b0c-b9d3-9f01f36f63ae admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:31:29.196018 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920201c4-8446-46a4-ae72-62bbd0b14e5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:31:29.201269 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920201c4-8446-46a4-ae72-62bbd0b14e5e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:31:29.202218 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920201c4-8446-46a4-ae72-62bbd0b14e5e admin admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:31:29.203794 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920201c4-8446-46a4-ae72-62bbd0b14e5e admin admin] PATH_INFO: `/v3/projects/63632453a279433b97469eac2f27b36a` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:31:29.211886 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-920201c4-8446-46a4-ae72-62bbd0b14e5e admin admin] RBAC: Authorizing `identity:delete_project(project_id=63632453a279433b97469eac2f27b36a)` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:31:29.221701 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-920201c4-8446-46a4-ae72-62bbd0b14e5e admin admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:31:29.326266 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea34754-8151-4b0c-b9d3-9f01f36f63ae admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:31:29.345006 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391ca0a1-342f-410e-b018-5b47d456826c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:31:29.346911 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391ca0a1-342f-410e-b018-5b47d456826c admin admin] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:31:29.347281 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391ca0a1-342f-410e-b018-5b47d456826c admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:31:29.351367 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391ca0a1-342f-410e-b018-5b47d456826c admin admin] PATH_INFO: `/v3/users` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:31:29.352558 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-391ca0a1-342f-410e-b018-5b47d456826c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:31:29.360813 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-391ca0a1-342f-410e-b018-5b47d456826c admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:31:29.472016 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 852/1708] 10.222.0.49 () {60 vars in 1143 bytes} [Wed Jun 15 10:31:29 2022] POST /identity/v3/users => generated 300 bytes in 297 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 10:31:29.472736 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391ca0a1-342f-410e-b018-5b47d456826c admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:31:29.484726 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-920201c4-8446-46a4-ae72-62bbd0b14e5e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '63632453a279433b97469eac2f27b36a', 'request_id': 'req-920201c4-8446-46a4-ae72-62bbd0b14e5e'} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:31:29.498309 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-920201c4-8446-46a4-ae72-62bbd0b14e5e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '63632453a279433b97469eac2f27b36a', 'request_id': 'req-920201c4-8446-46a4-ae72-62bbd0b14e5e'} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:31:29.510010 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-920201c4-8446-46a4-ae72-62bbd0b14e5e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '63632453a279433b97469eac2f27b36a', 'request_id': 'req-920201c4-8446-46a4-ae72-62bbd0b14e5e'} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:31:29.526168 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391ca0a1-342f-410e-b018-5b47d456826c admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:31:29.536254 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-920201c4-8446-46a4-ae72-62bbd0b14e5e admin admin] The token cache is being invalidate because project 63632453a279433b97469eac2f27b36a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80119) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 10:31:29.537472 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-920201c4-8446-46a4-ae72-62bbd0b14e5e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:31:29.539465 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-920201c4-8446-46a4-ae72-62bbd0b14e5e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:31:29.561573 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 857/1709] 10.222.0.49 () {58 vars in 1232 bytes} [Wed Jun 15 10:31:28 2022] DELETE /identity/v3/projects/63632453a279433b97469eac2f27b36a => generated 0 bytes in 733 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:31:29.567539 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdd54cf-c9e7-43bb-a488-74aca5ad9ed4 None None] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:31:29.567882 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdd54cf-c9e7-43bb-a488-74aca5ad9ed4 None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:31:29.568215 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdd54cf-c9e7-43bb-a488-74aca5ad9ed4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:31:29.649456 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.auth.core [None req-1bdd54cf-c9e7-43bb-a488-74aca5ad9ed4 None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:31:29.651923 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-1bdd54cf-c9e7-43bb-a488-74aca5ad9ed4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:31:29.817628 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 858/1710] 10.222.0.49 () {58 vars in 957 bytes} [Wed Jun 15 10:31:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 252 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 10:31:29.819365 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bdd54cf-c9e7-43bb-a488-74aca5ad9ed4 None None] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:31:29.826667 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-1bdd54cf-c9e7-43bb-a488-74aca5ad9ed4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:31:29.827019 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66db35d3-152f-4501-8126-543f64f1c7e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:31:29.827790 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66db35d3-152f-4501-8126-543f64f1c7e8 admin admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:31:29.828078 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66db35d3-152f-4501-8126-543f64f1c7e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:31:29.828406 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66db35d3-152f-4501-8126-543f64f1c7e8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:31:29.829218 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66db35d3-152f-4501-8126-543f64f1c7e8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:31:29.830842 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66db35d3-152f-4501-8126-543f64f1c7e8 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:31:29.853923 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 853/1711] 10.222.0.49 () {58 vars in 1121 bytes} [Wed Jun 15 10:31:29 2022] GET /identity/v3/roles => generated 1420 bytes in 382 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 10:31:29.857611 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a998fd-a3a2-4be1-8c48-9a921ab3e2eb None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:31:29.857906 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a998fd-a3a2-4be1-8c48-9a921ab3e2eb None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:31:29.858191 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a998fd-a3a2-4be1-8c48-9a921ab3e2eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:31:29.859704 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bdd54cf-c9e7-43bb-a488-74aca5ad9ed4 None None] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:31:29.872971 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb00f262-38bf-4c98-af2e-e4edcd11f919 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:31:29.875709 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb00f262-38bf-4c98-af2e-e4edcd11f919 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:31:29.881211 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb00f262-38bf-4c98-af2e-e4edcd11f919 admin admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:31:29.881640 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb00f262-38bf-4c98-af2e-e4edcd11f919 admin admin] PATH_INFO: `/v3/users/a6f02f41be834fe6b1ae803ae80ef6ee` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:31:29.887082 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb00f262-38bf-4c98-af2e-e4edcd11f919 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a6f02f41be834fe6b1ae803ae80ef6ee)` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:31:29.895191 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb00f262-38bf-4c98-af2e-e4edcd11f919 admin admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:31:29.907993 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-11a998fd-a3a2-4be1-8c48-9a921ab3e2eb None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:31:29.921135 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-11a998fd-a3a2-4be1-8c48-9a921ab3e2eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:31:29.963186 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 854/1712] 10.222.0.49 () {58 vars in 957 bytes} [Wed Jun 15 10:31:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:31:29.966369 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a998fd-a3a2-4be1-8c48-9a921ab3e2eb None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:31:29.972074 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-11a998fd-a3a2-4be1-8c48-9a921ab3e2eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:31:29.990149 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:31:30.036048 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a998fd-a3a2-4be1-8c48-9a921ab3e2eb None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:31:30.046541 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c1de78-451d-4237-a0a4-e927ca5d5934 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:31:30.049129 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c1de78-451d-4237-a0a4-e927ca5d5934 admin admin] REQUEST_METHOD: `PUT` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:31:30.050929 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c1de78-451d-4237-a0a4-e927ca5d5934 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:31:30.051964 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c1de78-451d-4237-a0a4-e927ca5d5934 admin admin] PATH_INFO: `/v3/projects/abb687a167db47dbb74b2654087c2cac/users/39eef429feea40b9b743cebbbe22d680/roles/8051174e7a7947d5b991ebd793c2e543` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:31:30.064752 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-cb00f262-38bf-4c98-af2e-e4edcd11f919 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a6f02f41be834fe6b1ae803ae80ef6ee'} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:31:30.084809 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-cb00f262-38bf-4c98-af2e-e4edcd11f919 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a6f02f41be834fe6b1ae803ae80ef6ee'} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:31:30.086763 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-cb00f262-38bf-4c98-af2e-e4edcd11f919 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a6f02f41be834fe6b1ae803ae80ef6ee'} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:31:30.089594 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-cb00f262-38bf-4c98-af2e-e4edcd11f919 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a6f02f41be834fe6b1ae803ae80ef6ee'} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:31:30.135021 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c1de78-451d-4237-a0a4-e927ca5d5934 admin admin] RBAC: Authorizing `identity:create_grant(project_id=abb687a167db47dbb74b2654087c2cac, user_id=39eef429feea40b9b743cebbbe22d680, role_id=8051174e7a7947d5b991ebd793c2e543)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:31:30.145741 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c1de78-451d-4237-a0a4-e927ca5d5934 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:31:30.157725 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-cb00f262-38bf-4c98-af2e-e4edcd11f919 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a6f02f41be834fe6b1ae803ae80ef6ee'} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:31:30.186840 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 855/1713] 10.222.0.49 () {60 vars in 1482 bytes} [Wed Jun 15 10:31:29 2022] PUT /identity/v3/projects/abb687a167db47dbb74b2654087c2cac/users/39eef429feea40b9b743cebbbe22d680/roles/8051174e7a7947d5b991ebd793c2e543 => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:31:30.190513 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c1de78-451d-4237-a0a4-e927ca5d5934 admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:31:30.198756 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-f2c1de78-451d-4237-a0a4-e927ca5d5934 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:31:30.209827 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 859/1714] 10.222.0.49 () {58 vars in 1223 bytes} [Wed Jun 15 10:31:29 2022] DELETE /identity/v3/users/a6f02f41be834fe6b1ae803ae80ef6ee => generated 0 bytes in 388 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:31:30.209827 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb00f262-38bf-4c98-af2e-e4edcd11f919 admin admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:31:30.212346 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-cb00f262-38bf-4c98-af2e-e4edcd11f919 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:31:30.246313 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:31:30.411739 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c1de78-451d-4237-a0a4-e927ca5d5934 admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:31:30.538451 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb00f262-38bf-4c98-af2e-e4edcd11f919 admin admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:31:30.561722 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2700d7-2d83-49ad-b12b-7686a8589d0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:31:30.565249 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2700d7-2d83-49ad-b12b-7686a8589d0c admin admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:31:30.565550 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2700d7-2d83-49ad-b12b-7686a8589d0c admin admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:31:30.566200 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2700d7-2d83-49ad-b12b-7686a8589d0c admin admin] PATH_INFO: `/v3/roles` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:31:30.566741 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2700d7-2d83-49ad-b12b-7686a8589d0c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:31:30.568465 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2700d7-2d83-49ad-b12b-7686a8589d0c admin admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:31:30.612976 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 860/1715] 10.222.0.49 () {58 vars in 1121 bytes} [Wed Jun 15 10:31:30 2022] GET /identity/v3/roles => generated 1420 bytes in 405 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 10:31:30.624190 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbc65fd-13ef-460e-adb7-2714df5e4514 None None] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:31:30.624489 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbc65fd-13ef-460e-adb7-2714df5e4514 None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:31:30.624776 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbc65fd-13ef-460e-adb7-2714df5e4514 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:31:30.674190 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bea6e8-2f67-4162-9947-a501a42957e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:31:30.674956 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bea6e8-2f67-4162-9947-a501a42957e4 admin admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:31:30.675974 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bea6e8-2f67-4162-9947-a501a42957e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:31:30.676278 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bea6e8-2f67-4162-9947-a501a42957e4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:31:30.677730 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4bea6e8-2f67-4162-9947-a501a42957e4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:31:30.678042 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4bea6e8-2f67-4162-9947-a501a42957e4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:31:30.683147 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4bea6e8-2f67-4162-9947-a501a42957e4 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:31:30.710842 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.auth.core [None req-bcbc65fd-13ef-460e-adb7-2714df5e4514 None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:31:30.717255 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-bcbc65fd-13ef-460e-adb7-2714df5e4514 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:31:30.718688 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 856/1716] 10.222.0.49 () {58 vars in 1152 bytes} [Wed Jun 15 10:31:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 528 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:31:30.719952 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bea6e8-2f67-4162-9947-a501a42957e4 admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:31:30.775418 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 861/1717] 10.222.0.49 () {58 vars in 957 bytes} [Wed Jun 15 10:31:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:31:30.776311 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcbc65fd-13ef-460e-adb7-2714df5e4514 None None] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:31:30.802200 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bea6e8-2f67-4162-9947-a501a42957e4 admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:31:30.825767 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcbc65fd-13ef-460e-adb7-2714df5e4514 None None] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:31:30.840184 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c25504b-ca7a-4a3b-a484-0a23c4e47ffb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:31:30.840820 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c25504b-ca7a-4a3b-a484-0a23c4e47ffb admin admin] REQUEST_METHOD: `PUT` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:31:30.841203 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c25504b-ca7a-4a3b-a484-0a23c4e47ffb admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:31:30.841490 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c25504b-ca7a-4a3b-a484-0a23c4e47ffb admin admin] PATH_INFO: `/v3/projects/abb687a167db47dbb74b2654087c2cac/users/39eef429feea40b9b743cebbbe22d680/roles/8051174e7a7947d5b991ebd793c2e543` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:31:30.865946 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c25504b-ca7a-4a3b-a484-0a23c4e47ffb admin admin] RBAC: Authorizing `identity:create_grant(project_id=abb687a167db47dbb74b2654087c2cac, user_id=39eef429feea40b9b743cebbbe22d680, role_id=8051174e7a7947d5b991ebd793c2e543)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:31:30.874365 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aed3603-4b6f-45cb-a535-e3a02678c631 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:31:30.875078 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aed3603-4b6f-45cb-a535-e3a02678c631 admin admin] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:31:30.875388 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aed3603-4b6f-45cb-a535-e3a02678c631 admin admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:31:30.875670 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aed3603-4b6f-45cb-a535-e3a02678c631 admin admin] PATH_INFO: `/v3/projects` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:31:30.876375 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aed3603-4b6f-45cb-a535-e3a02678c631 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:31:30.878308 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aed3603-4b6f-45cb-a535-e3a02678c631 admin admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:31:30.880131 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c25504b-ca7a-4a3b-a484-0a23c4e47ffb admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:31:30.903622 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 857/1718] 10.222.0.49 () {60 vars in 1482 bytes} [Wed Jun 15 10:31:30 2022] PUT /identity/v3/projects/abb687a167db47dbb74b2654087c2cac/users/39eef429feea40b9b743cebbbe22d680/roles/8051174e7a7947d5b991ebd793c2e543 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:31:30.925375 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c25504b-ca7a-4a3b-a484-0a23c4e47ffb admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:31:30.941005 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-9c25504b-ca7a-4a3b-a484-0a23c4e47ffb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:31:30.980094 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 862/1719] 10.222.0.49 () {60 vars in 1152 bytes} [Wed Jun 15 10:31:30 2022] POST /identity/v3/projects => generated 373 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 10:31:30.982160 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14eb4bb-c776-4076-8680-bb57c0855a18 None None] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:31:30.982514 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14eb4bb-c776-4076-8680-bb57c0855a18 None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:31:30.982828 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14eb4bb-c776-4076-8680-bb57c0855a18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:31:31.100971 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.auth.core [None req-a14eb4bb-c776-4076-8680-bb57c0855a18 None None] MFA Rules not processed for user `39eef429feea40b9b743cebbbe22d680`. Rule list: `[]` (Enabled: `True`). {{(pid=80119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:31:31.103746 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-a14eb4bb-c776-4076-8680-bb57c0855a18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:31:31.174021 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c25504b-ca7a-4a3b-a484-0a23c4e47ffb admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:31:31.302020 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:DELETE=8,keystone:SELECT=90,keystone:INSERT=2 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:31:31.352166 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0d320d-7621-497d-80b0-e969a4076bc3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0cb44a17d205431793e9a8791e477a32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '559b3e713f5844b6ae37151d56dd419f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:31:31.353507 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0d320d-7621-497d-80b0-e969a4076bc3 service neutron] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:31:31.353798 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0d320d-7621-497d-80b0-e969a4076bc3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:31:31.354089 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0d320d-7621-497d-80b0-e969a4076bc3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:31:31.410610 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 863/1720] 10.222.0.49 () {58 vars in 957 bytes} [Wed Jun 15 10:31:30 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 429 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:31:31.414566 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14eb4bb-c776-4076-8680-bb57c0855a18 None None] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:31:31.427570 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:31:31.661781 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0d320d-7621-497d-80b0-e969a4076bc3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:31:31.667502 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0d320d-7621-497d-80b0-e969a4076bc3 service neutron] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:31:31.694703 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 858/1721] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Jun 15 10:31:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 791 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:31:31.697068 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0d320d-7621-497d-80b0-e969a4076bc3 service neutron] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:31:31.699985 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-8a0d320d-7621-497d-80b0-e969a4076bc3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:31:31.760647 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14eb4bb-c776-4076-8680-bb57c0855a18 None None] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:31:31.787290 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a1bd7e-8e97-4eb3-9679-68e465ed06e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:31:31.788050 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a1bd7e-8e97-4eb3-9679-68e465ed06e3 admin admin] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:31:31.788335 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a1bd7e-8e97-4eb3-9679-68e465ed06e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:31:31.788607 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a1bd7e-8e97-4eb3-9679-68e465ed06e3 admin admin] PATH_INFO: `/v3/users` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:31:31.791669 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a1bd7e-8e97-4eb3-9679-68e465ed06e3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:31:31.794777 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a1bd7e-8e97-4eb3-9679-68e465ed06e3 admin admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:31:31.850157 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 864/1722] 10.222.0.49 () {60 vars in 1143 bytes} [Wed Jun 15 10:31:31 2022] POST /identity/v3/users => generated 287 bytes in 439 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 10:31:31.851605 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a1bd7e-8e97-4eb3-9679-68e465ed06e3 admin admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:31:31.894716 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0d320d-7621-497d-80b0-e969a4076bc3 service neutron] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:31:31.903045 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a1bd7e-8e97-4eb3-9679-68e465ed06e3 admin admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:31:31.906822 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd38e3e-6686-4a5b-a3fc-a4a433025459 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20d1327553404b80aecee87c78fcb52b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '559b3e713f5844b6ae37151d56dd419f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:31:31.907826 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd38e3e-6686-4a5b-a3fc-a4a433025459 service cinder] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:31:31.908021 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd38e3e-6686-4a5b-a3fc-a4a433025459 service cinder] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:31:31.908604 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd38e3e-6686-4a5b-a3fc-a4a433025459 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:31:31.927523 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34171bb7-7f6d-40ad-a9f2-1f4eb9c18f55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:31:31.931775 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd38e3e-6686-4a5b-a3fc-a4a433025459 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:31:31.937999 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd38e3e-6686-4a5b-a3fc-a4a433025459 service cinder] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:31:31.960343 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34171bb7-7f6d-40ad-a9f2-1f4eb9c18f55 admin admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:31:31.961488 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34171bb7-7f6d-40ad-a9f2-1f4eb9c18f55 admin admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:31:31.962616 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34171bb7-7f6d-40ad-a9f2-1f4eb9c18f55 admin admin] PATH_INFO: `/v3/roles` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:31:31.965555 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34171bb7-7f6d-40ad-a9f2-1f4eb9c18f55 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:31:31.971680 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 859/1723] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Jun 15 10:31:31 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 275 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:31:31.972846 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd38e3e-6686-4a5b-a3fc-a4a433025459 service cinder] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:31:31.975882 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34171bb7-7f6d-40ad-a9f2-1f4eb9c18f55 admin admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:31:31.977408 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-2cd38e3e-6686-4a5b-a3fc-a4a433025459 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:31:32.033404 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 865/1724] 10.222.0.49 () {58 vars in 1121 bytes} [Wed Jun 15 10:31:31 2022] GET /identity/v3/roles => generated 1420 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 10:31:32.035561 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd38e3e-6686-4a5b-a3fc-a4a433025459 service cinder] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:31:32.041241 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34171bb7-7f6d-40ad-a9f2-1f4eb9c18f55 admin admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:31:32.046722 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59bd204-0bea-48a8-ab85-9b74cadfcf94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:31:32.047776 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59bd204-0bea-48a8-ab85-9b74cadfcf94 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:31:32.047983 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59bd204-0bea-48a8-ab85-9b74cadfcf94 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:31:32.048204 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59bd204-0bea-48a8-ab85-9b74cadfcf94 admin admin] PATH_INFO: `/v3/users/f445676a7f1846acb2964c6a5c7ee0a0` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:31:32.051332 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e59bd204-0bea-48a8-ab85-9b74cadfcf94 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f445676a7f1846acb2964c6a5c7ee0a0)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:31:32.063777 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e59bd204-0bea-48a8-ab85-9b74cadfcf94 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:31:32.085601 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34171bb7-7f6d-40ad-a9f2-1f4eb9c18f55 admin admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:31:32.107665 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41cc627f-2378-452f-b193-522946ba7af4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:31:32.112268 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cc627f-2378-452f-b193-522946ba7af4 admin admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:31:32.112571 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cc627f-2378-452f-b193-522946ba7af4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:31:32.112968 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cc627f-2378-452f-b193-522946ba7af4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:31:32.114811 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41cc627f-2378-452f-b193-522946ba7af4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:31:32.121185 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41cc627f-2378-452f-b193-522946ba7af4 admin admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:31:32.206896 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 866/1725] 10.222.0.49 () {58 vars in 1121 bytes} [Wed Jun 15 10:31:32 2022] GET /identity/v3/roles => generated 1420 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 10:31:32.220231 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41cc627f-2378-452f-b193-522946ba7af4 admin admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:31:32.242115 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-e59bd204-0bea-48a8-ab85-9b74cadfcf94 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f445676a7f1846acb2964c6a5c7ee0a0'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:31:32.271740 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-e59bd204-0bea-48a8-ab85-9b74cadfcf94 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f445676a7f1846acb2964c6a5c7ee0a0'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:31:32.272925 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-e59bd204-0bea-48a8-ab85-9b74cadfcf94 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f445676a7f1846acb2964c6a5c7ee0a0'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:31:32.279531 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-e59bd204-0bea-48a8-ab85-9b74cadfcf94 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f445676a7f1846acb2964c6a5c7ee0a0'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:31:32.288706 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41cc627f-2378-452f-b193-522946ba7af4 admin admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:31:32.308877 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-e59bd204-0bea-48a8-ab85-9b74cadfcf94 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f445676a7f1846acb2964c6a5c7ee0a0'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:31:32.317831 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eca0a64-fd28-4878-92b1-91e0d47307b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:31:32.321661 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eca0a64-fd28-4878-92b1-91e0d47307b7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:31:32.326363 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eca0a64-fd28-4878-92b1-91e0d47307b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:31:32.326660 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eca0a64-fd28-4878-92b1-91e0d47307b7 admin admin] PATH_INFO: `/v3/projects/6a7f760928d048fd8d39c90c7796515d/users/5a4b9994ac914cbcab64e37132baac9a/roles/6876e57cc51c448097ab109d35b2a3da` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:31:32.375397 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 860/1726] 10.222.0.49 () {58 vars in 1223 bytes} [Wed Jun 15 10:31:31 2022] DELETE /identity/v3/users/f445676a7f1846acb2964c6a5c7ee0a0 => generated 0 bytes in 400 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:31:32.379684 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59bd204-0bea-48a8-ab85-9b74cadfcf94 admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:31:32.385165 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-e59bd204-0bea-48a8-ab85-9b74cadfcf94 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:31:32.393725 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eca0a64-fd28-4878-92b1-91e0d47307b7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6a7f760928d048fd8d39c90c7796515d, user_id=5a4b9994ac914cbcab64e37132baac9a, role_id=6876e57cc51c448097ab109d35b2a3da)` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:31:32.409702 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eca0a64-fd28-4878-92b1-91e0d47307b7 admin admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:31:32.470856 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 867/1727] 10.222.0.49 () {60 vars in 1482 bytes} [Wed Jun 15 10:31:32 2022] PUT /identity/v3/projects/6a7f760928d048fd8d39c90c7796515d/users/5a4b9994ac914cbcab64e37132baac9a/roles/6876e57cc51c448097ab109d35b2a3da => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:31:32.485197 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0de630-2ac0-4439-821b-ea1495dcc0ce None None] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:31:32.488822 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0de630-2ac0-4439-821b-ea1495dcc0ce None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:31:32.489320 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0de630-2ac0-4439-821b-ea1495dcc0ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:31:32.505437 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:31:32.581440 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.auth.core [None req-da0de630-2ac0-4439-821b-ea1495dcc0ce None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:31:32.588123 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-da0de630-2ac0-4439-821b-ea1495dcc0ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:31:32.629784 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59bd204-0bea-48a8-ab85-9b74cadfcf94 admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:31:32.817814 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 868/1728] 10.222.0.49 () {58 vars in 957 bytes} [Wed Jun 15 10:31:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 346 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:31:32.820388 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a702b2-1e9b-418c-95e4-d2554602713d None None] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:31:32.821147 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a702b2-1e9b-418c-95e4-d2554602713d None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:31:32.821538 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a702b2-1e9b-418c-95e4-d2554602713d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:31:32.825869 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b62bb71-2a2c-4cd6-bb33-3f8a206ec58d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:31:32.826894 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b62bb71-2a2c-4cd6-bb33-3f8a206ec58d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:31:32.827322 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b62bb71-2a2c-4cd6-bb33-3f8a206ec58d admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:31:32.827730 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b62bb71-2a2c-4cd6-bb33-3f8a206ec58d admin admin] PATH_INFO: `/v3/users/1f73691a6eac4f009faa844945a797dd` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:31:32.829805 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b62bb71-2a2c-4cd6-bb33-3f8a206ec58d admin admin] RBAC: Authorizing `identity:delete_user(user_id=1f73691a6eac4f009faa844945a797dd)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:31:32.835024 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b62bb71-2a2c-4cd6-bb33-3f8a206ec58d admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:31:32.891295 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.auth.core [None req-a3a702b2-1e9b-418c-95e4-d2554602713d None None] MFA Rules not processed for user `5a4b9994ac914cbcab64e37132baac9a`. Rule list: `[]` (Enabled: `True`). {{(pid=80119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:31:32.892548 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-a3a702b2-1e9b-418c-95e4-d2554602713d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:31:32.988249 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:31:33.001225 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-2b62bb71-2a2c-4cd6-bb33-3f8a206ec58d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1f73691a6eac4f009faa844945a797dd'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:31:33.027825 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-2b62bb71-2a2c-4cd6-bb33-3f8a206ec58d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1f73691a6eac4f009faa844945a797dd'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:31:33.031542 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-2b62bb71-2a2c-4cd6-bb33-3f8a206ec58d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1f73691a6eac4f009faa844945a797dd'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:31:33.033041 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-2b62bb71-2a2c-4cd6-bb33-3f8a206ec58d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1f73691a6eac4f009faa844945a797dd'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:31:33.065307 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-2b62bb71-2a2c-4cd6-bb33-3f8a206ec58d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1f73691a6eac4f009faa844945a797dd'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:31:33.111932 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 861/1729] 10.222.0.49 () {58 vars in 1223 bytes} [Wed Jun 15 10:31:32 2022] DELETE /identity/v3/users/1f73691a6eac4f009faa844945a797dd => generated 0 bytes in 739 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:31:33.118946 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b62bb71-2a2c-4cd6-bb33-3f8a206ec58d admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:31:33.132268 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-2b62bb71-2a2c-4cd6-bb33-3f8a206ec58d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:31:33.158638 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 869/1730] 10.222.0.49 () {58 vars in 957 bytes} [Wed Jun 15 10:31:32 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 340 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:31:33.161020 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24e1a62-868c-4c47-bc86-5195852b3c31 None None] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:31:33.162664 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24e1a62-868c-4c47-bc86-5195852b3c31 None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:31:33.163135 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24e1a62-868c-4c47-bc86-5195852b3c31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:31:33.202393 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.auth.core [None req-f24e1a62-868c-4c47-bc86-5195852b3c31 None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:31:33.203956 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-f24e1a62-868c-4c47-bc86-5195852b3c31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:31:33.294100 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b62bb71-2a2c-4cd6-bb33-3f8a206ec58d admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:31:33.302867 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303de53f-b520-4f72-ba3c-d7ff648c93a3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0cb44a17d205431793e9a8791e477a32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '559b3e713f5844b6ae37151d56dd419f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:31:33.317801 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303de53f-b520-4f72-ba3c-d7ff648c93a3 service neutron] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:31:33.318241 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303de53f-b520-4f72-ba3c-d7ff648c93a3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:31:33.318781 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303de53f-b520-4f72-ba3c-d7ff648c93a3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:31:33.326048 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-303de53f-b520-4f72-ba3c-d7ff648c93a3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:31:33.351503 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:31:33.434390 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 870/1731] 10.222.0.49 () {58 vars in 957 bytes} [Wed Jun 15 10:31:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 274 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:31:33.437288 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24e1a62-868c-4c47-bc86-5195852b3c31 None None] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:31:33.442948 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-f24e1a62-868c-4c47-bc86-5195852b3c31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:31:33.674821 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-303de53f-b520-4f72-ba3c-d7ff648c93a3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:31:33.682899 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-303de53f-b520-4f72-ba3c-d7ff648c93a3 service neutron] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:31:33.721856 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 862/1732] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Jun 15 10:31:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 609 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:31:33.723884 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a16646-bb6f-47cb-981c-fd2fc7a92d7e None None] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:31:33.724764 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a16646-bb6f-47cb-981c-fd2fc7a92d7e None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:31:33.724876 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a16646-bb6f-47cb-981c-fd2fc7a92d7e None None] PATH_INFO: `/` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:31:33.726631 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 863/1733] 10.222.0.49 () {54 vars in 921 bytes} [Wed Jun 15 10:31:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:31:33.736987 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a16646-bb6f-47cb-981c-fd2fc7a92d7e None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:31:33.747364 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-d6a16646-bb6f-47cb-981c-fd2fc7a92d7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:31:33.807323 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a16646-bb6f-47cb-981c-fd2fc7a92d7e None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:31:33.815072 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a593a3-d173-42d9-af74-a7aba18c86b9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0cb44a17d205431793e9a8791e477a32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '559b3e713f5844b6ae37151d56dd419f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:31:33.815786 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a593a3-d173-42d9-af74-a7aba18c86b9 service neutron] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:31:33.816076 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a593a3-d173-42d9-af74-a7aba18c86b9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:31:33.816442 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a593a3-d173-42d9-af74-a7aba18c86b9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:31:33.831424 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a593a3-d173-42d9-af74-a7aba18c86b9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:31:33.833011 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a593a3-d173-42d9-af74-a7aba18c86b9 service neutron] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:31:33.848641 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 864/1734] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Jun 15 10:31:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:31:33.912582 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24e1a62-868c-4c47-bc86-5195852b3c31 None None] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:31:33.938249 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1229432d-4e52-4d40-ad9d-047f030e59bc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ac54528370d4747b6e8777fba9d0c82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '559b3e713f5844b6ae37151d56dd419f', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:31:33.939516 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1229432d-4e52-4d40-ad9d-047f030e59bc service nova] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:31:33.940143 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1229432d-4e52-4d40-ad9d-047f030e59bc service nova] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:31:33.940442 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1229432d-4e52-4d40-ad9d-047f030e59bc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:31:33.975438 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954cf585-e9ed-44ab-9dd3-e95be22155fd None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:31:33.975733 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954cf585-e9ed-44ab-9dd3-e95be22155fd None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:31:33.976022 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954cf585-e9ed-44ab-9dd3-e95be22155fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:31:34.050397 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-954cf585-e9ed-44ab-9dd3-e95be22155fd None None] MFA Rules not processed for user `5ac54528370d4747b6e8777fba9d0c82`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:31:34.051782 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-954cf585-e9ed-44ab-9dd3-e95be22155fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:31:34.095947 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 865/1735] 10.222.0.49 () {58 vars in 1018 bytes} [Wed Jun 15 10:31:33 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 10:31:34.144390 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1229432d-4e52-4d40-ad9d-047f030e59bc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:31:34.146392 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1229432d-4e52-4d40-ad9d-047f030e59bc service nova] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:31:34.153814 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=100 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:31:34.162395 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 871/1736] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Jun 15 10:31:33 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 727 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:31:34.233351 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954cf585-e9ed-44ab-9dd3-e95be22155fd None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:31:34.235041 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-954cf585-e9ed-44ab-9dd3-e95be22155fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:31:34.258896 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954cf585-e9ed-44ab-9dd3-e95be22155fd None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:31:34.271178 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7cc6925-7ba0-4bf2-867d-3b6b61f70301 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:31:34.272746 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cc6925-7ba0-4bf2-867d-3b6b61f70301 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:31:34.273295 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cc6925-7ba0-4bf2-867d-3b6b61f70301 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:31:34.273873 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cc6925-7ba0-4bf2-867d-3b6b61f70301 admin admin] PATH_INFO: `/v3/projects/086345a5a5b8414bb44888eb24463686` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:31:34.276359 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7cc6925-7ba0-4bf2-867d-3b6b61f70301 admin admin] RBAC: Authorizing `identity:delete_project(project_id=086345a5a5b8414bb44888eb24463686)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:31:34.279234 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7cc6925-7ba0-4bf2-867d-3b6b61f70301 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:31:34.398426 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-d7cc6925-7ba0-4bf2-867d-3b6b61f70301 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '086345a5a5b8414bb44888eb24463686', 'request_id': 'req-d7cc6925-7ba0-4bf2-867d-3b6b61f70301'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:31:34.421360 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-d7cc6925-7ba0-4bf2-867d-3b6b61f70301 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '086345a5a5b8414bb44888eb24463686', 'request_id': 'req-d7cc6925-7ba0-4bf2-867d-3b6b61f70301'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:31:34.433178 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-d7cc6925-7ba0-4bf2-867d-3b6b61f70301 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '086345a5a5b8414bb44888eb24463686', 'request_id': 'req-d7cc6925-7ba0-4bf2-867d-3b6b61f70301'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:31:34.467498 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-d7cc6925-7ba0-4bf2-867d-3b6b61f70301 admin admin] The token cache is being invalidate because project 086345a5a5b8414bb44888eb24463686 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80120) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 10:31:34.467879 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-d7cc6925-7ba0-4bf2-867d-3b6b61f70301 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:31:34.470545 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-d7cc6925-7ba0-4bf2-867d-3b6b61f70301 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:31:34.472289 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 866/1737] 10.222.0.49 () {58 vars in 1232 bytes} [Wed Jun 15 10:31:34 2022] DELETE /identity/v3/projects/086345a5a5b8414bb44888eb24463686 => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:31:34.476186 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1229432d-4e52-4d40-ad9d-047f030e59bc service nova] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:31:34.497065 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1229432d-4e52-4d40-ad9d-047f030e59bc service nova] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:31:34.795327 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9326ae38-ab89-4a75-bf8e-05ae4a4b6b3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:31:34.796177 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9326ae38-ab89-4a75-bf8e-05ae4a4b6b3d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:31:34.796486 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9326ae38-ab89-4a75-bf8e-05ae4a4b6b3d admin admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:31:34.796765 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9326ae38-ab89-4a75-bf8e-05ae4a4b6b3d admin admin] PATH_INFO: `/v3/users/ebd5a5c7cd594a69aec95ba00d4f5612` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:31:34.801597 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9326ae38-ab89-4a75-bf8e-05ae4a4b6b3d admin admin] RBAC: Authorizing `identity:delete_user(user_id=ebd5a5c7cd594a69aec95ba00d4f5612)` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:31:34.807111 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9326ae38-ab89-4a75-bf8e-05ae4a4b6b3d admin admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:31:34.934887 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-9326ae38-ab89-4a75-bf8e-05ae4a4b6b3d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ebd5a5c7cd594a69aec95ba00d4f5612'} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:31:34.971747 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-9326ae38-ab89-4a75-bf8e-05ae4a4b6b3d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ebd5a5c7cd594a69aec95ba00d4f5612'} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:31:34.974388 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-9326ae38-ab89-4a75-bf8e-05ae4a4b6b3d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ebd5a5c7cd594a69aec95ba00d4f5612'} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:31:34.976502 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-9326ae38-ab89-4a75-bf8e-05ae4a4b6b3d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ebd5a5c7cd594a69aec95ba00d4f5612'} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:31:34.999762 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-9326ae38-ab89-4a75-bf8e-05ae4a4b6b3d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ebd5a5c7cd594a69aec95ba00d4f5612'} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:31:35.049977 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 872/1738] 10.222.0.49 () {58 vars in 1223 bytes} [Wed Jun 15 10:31:34 2022] DELETE /identity/v3/users/ebd5a5c7cd594a69aec95ba00d4f5612 => generated 0 bytes in 574 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:31:35.326907 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7cc6925-7ba0-4bf2-867d-3b6b61f70301 admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:31:35.328992 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-d7cc6925-7ba0-4bf2-867d-3b6b61f70301 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:31:35.385766 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b959e5f5-81c6-4b0b-bb58-eb5f1827cca6 None None] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:31:35.386029 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b959e5f5-81c6-4b0b-bb58-eb5f1827cca6 None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:31:35.386335 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b959e5f5-81c6-4b0b-bb58-eb5f1827cca6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:31:35.455327 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.auth.core [None req-b959e5f5-81c6-4b0b-bb58-eb5f1827cca6 None None] MFA Rules not processed for user `14b720a2d0854d218272390205887100`. Rule list: `[]` (Enabled: `True`). {{(pid=80119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:31:35.456620 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-b959e5f5-81c6-4b0b-bb58-eb5f1827cca6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:31:35.543623 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:31:35.584469 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7cc6925-7ba0-4bf2-867d-3b6b61f70301 admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:31:35.619702 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 873/1739] 10.222.0.49 () {58 vars in 957 bytes} [Wed Jun 15 10:31:35 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 235 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 10:31:35.784491 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9de46ea-e99f-490a-b7d4-926ada8f18d5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ac54528370d4747b6e8777fba9d0c82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '559b3e713f5844b6ae37151d56dd419f', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:31:35.785408 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9de46ea-e99f-490a-b7d4-926ada8f18d5 service nova] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:31:35.785855 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9de46ea-e99f-490a-b7d4-926ada8f18d5 service nova] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:31:35.786232 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9de46ea-e99f-490a-b7d4-926ada8f18d5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:31:35.792024 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-d9de46ea-e99f-490a-b7d4-926ada8f18d5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:31:35.839818 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9de46ea-e99f-490a-b7d4-926ada8f18d5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:31:35.841541 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9de46ea-e99f-490a-b7d4-926ada8f18d5 service nova] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:31:35.890774 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 867/1740] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Jun 15 10:31:35 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 564 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:31:36.160506 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b959e5f5-81c6-4b0b-bb58-eb5f1827cca6 None None] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:31:36.163099 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-b959e5f5-81c6-4b0b-bb58-eb5f1827cca6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:31:36.224227 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b959e5f5-81c6-4b0b-bb58-eb5f1827cca6 None None] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:31:36.249273 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d816d45b-d3ea-4cf8-931e-f87ec8dfd2eb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ac54528370d4747b6e8777fba9d0c82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '559b3e713f5844b6ae37151d56dd419f', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:31:36.251831 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d816d45b-d3ea-4cf8-931e-f87ec8dfd2eb service nova] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:31:36.255324 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d816d45b-d3ea-4cf8-931e-f87ec8dfd2eb service nova] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:31:36.255776 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d816d45b-d3ea-4cf8-931e-f87ec8dfd2eb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:31:36.281705 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:31:36.287920 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d816d45b-d3ea-4cf8-931e-f87ec8dfd2eb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:31:36.292946 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d816d45b-d3ea-4cf8-931e-f87ec8dfd2eb service nova] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:31:36.329648 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 874/1741] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Jun 15 10:31:36 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:31:36.588810 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f9dfe8-c4b6-4f9f-bd5c-ac84513c2177 None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:31:36.589308 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f9dfe8-c4b6-4f9f-bd5c-ac84513c2177 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:31:36.589393 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f9dfe8-c4b6-4f9f-bd5c-ac84513c2177 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:31:36.673742 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-b1f9dfe8-c4b6-4f9f-bd5c-ac84513c2177 None None] MFA Rules not processed for user `cca7cb5e7620446ebae91b1fe12bbd24`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:31:36.676955 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-b1f9dfe8-c4b6-4f9f-bd5c-ac84513c2177 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:31:36.842138 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d816d45b-d3ea-4cf8-931e-f87ec8dfd2eb service nova] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:31:36.844033 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-d816d45b-d3ea-4cf8-931e-f87ec8dfd2eb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:31:36.912950 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 868/1742] 10.222.0.49 () {58 vars in 957 bytes} [Wed Jun 15 10:31:36 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 325 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:31:36.928073 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f9dfe8-c4b6-4f9f-bd5c-ac84513c2177 None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:31:36.956038 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f9dfe8-c4b6-4f9f-bd5c-ac84513c2177 None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:31:36.987797 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22a9848-bf39-48cc-8dde-7f0803c414d6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ac54528370d4747b6e8777fba9d0c82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '559b3e713f5844b6ae37151d56dd419f', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:31:36.988483 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22a9848-bf39-48cc-8dde-7f0803c414d6 service nova] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:31:36.988773 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22a9848-bf39-48cc-8dde-7f0803c414d6 service nova] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:31:36.989056 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22a9848-bf39-48cc-8dde-7f0803c414d6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:31:37.017071 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22a9848-bf39-48cc-8dde-7f0803c414d6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:31:37.018709 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22a9848-bf39-48cc-8dde-7f0803c414d6 service nova] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:31:37.050137 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 869/1743] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Jun 15 10:31:36 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 10:31:37.050325 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d816d45b-d3ea-4cf8-931e-f87ec8dfd2eb service nova] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:31:37.083246 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5622afd0-574c-4c0e-80dd-8273fe3cd869 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:31:37.085460 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5622afd0-574c-4c0e-80dd-8273fe3cd869 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:31:37.100816 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5622afd0-574c-4c0e-80dd-8273fe3cd869 admin admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:31:37.106591 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5622afd0-574c-4c0e-80dd-8273fe3cd869 admin admin] PATH_INFO: `/v3/projects/8b42eec6ead74457af76efc70244987c` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:31:37.108878 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5622afd0-574c-4c0e-80dd-8273fe3cd869 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8b42eec6ead74457af76efc70244987c)` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:31:37.111246 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5622afd0-574c-4c0e-80dd-8273fe3cd869 admin admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:31:37.362432 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-5622afd0-574c-4c0e-80dd-8273fe3cd869 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8b42eec6ead74457af76efc70244987c', 'request_id': 'req-5622afd0-574c-4c0e-80dd-8273fe3cd869'} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:31:37.388915 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-5622afd0-574c-4c0e-80dd-8273fe3cd869 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8b42eec6ead74457af76efc70244987c', 'request_id': 'req-5622afd0-574c-4c0e-80dd-8273fe3cd869'} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:31:37.403760 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-5622afd0-574c-4c0e-80dd-8273fe3cd869 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8b42eec6ead74457af76efc70244987c', 'request_id': 'req-5622afd0-574c-4c0e-80dd-8273fe3cd869'} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:31:37.434481 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-5622afd0-574c-4c0e-80dd-8273fe3cd869 admin admin] The token cache is being invalidate because project 8b42eec6ead74457af76efc70244987c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80119) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 10:31:37.435216 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-5622afd0-574c-4c0e-80dd-8273fe3cd869 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:31:37.437043 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-5622afd0-574c-4c0e-80dd-8273fe3cd869 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:31:37.439549 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 875/1744] 10.222.0.49 () {58 vars in 1232 bytes} [Wed Jun 15 10:31:36 2022] DELETE /identity/v3/projects/8b42eec6ead74457af76efc70244987c => generated 0 bytes in 598 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:31:37.444288 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0f8308-9fa5-4f07-83ed-4e9d7bf70dd3 None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:31:37.444864 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0f8308-9fa5-4f07-83ed-4e9d7bf70dd3 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:31:37.445560 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0f8308-9fa5-4f07-83ed-4e9d7bf70dd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:31:37.487504 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-7f0f8308-9fa5-4f07-83ed-4e9d7bf70dd3 None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:31:37.499715 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-7f0f8308-9fa5-4f07-83ed-4e9d7bf70dd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:31:37.520950 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5622afd0-574c-4c0e-80dd-8273fe3cd869 admin admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:31:37.524155 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-5622afd0-574c-4c0e-80dd-8273fe3cd869 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:31:37.843212 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 870/1745] 10.222.0.49 () {58 vars in 957 bytes} [Wed Jun 15 10:31:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 400 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:31:37.851560 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487bb287-2b87-4a33-9bfa-58acb124dc9e None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:31:37.853706 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487bb287-2b87-4a33-9bfa-58acb124dc9e None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:31:37.854132 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487bb287-2b87-4a33-9bfa-58acb124dc9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:31:37.861379 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5622afd0-574c-4c0e-80dd-8273fe3cd869 admin admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:31:37.899154 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e61367c-294d-41b8-b106-1f6feb6364d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:31:37.908262 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:31:37.908854 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e61367c-294d-41b8-b106-1f6feb6364d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:31:37.909856 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e61367c-294d-41b8-b106-1f6feb6364d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:31:37.910373 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e61367c-294d-41b8-b106-1f6feb6364d5 admin admin] PATH_INFO: `/v3/projects/ef878af2a9184c309f244b3be501fd73` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:31:37.913682 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e61367c-294d-41b8-b106-1f6feb6364d5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ef878af2a9184c309f244b3be501fd73)` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:31:37.918435 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e61367c-294d-41b8-b106-1f6feb6364d5 admin admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:31:37.925124 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-487bb287-2b87-4a33-9bfa-58acb124dc9e None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:31:37.928193 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-487bb287-2b87-4a33-9bfa-58acb124dc9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:31:37.968617 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 871/1746] 10.222.0.49 () {58 vars in 957 bytes} [Wed Jun 15 10:31:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 10:31:37.969774 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487bb287-2b87-4a33-9bfa-58acb124dc9e None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:31:37.972167 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-487bb287-2b87-4a33-9bfa-58acb124dc9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:31:38.023317 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:31:38.024035 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487bb287-2b87-4a33-9bfa-58acb124dc9e None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:31:38.060849 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-3e61367c-294d-41b8-b106-1f6feb6364d5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ef878af2a9184c309f244b3be501fd73', 'request_id': 'req-3e61367c-294d-41b8-b106-1f6feb6364d5'} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:31:38.076700 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-3e61367c-294d-41b8-b106-1f6feb6364d5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ef878af2a9184c309f244b3be501fd73', 'request_id': 'req-3e61367c-294d-41b8-b106-1f6feb6364d5'} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:31:38.089841 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-3e61367c-294d-41b8-b106-1f6feb6364d5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ef878af2a9184c309f244b3be501fd73', 'request_id': 'req-3e61367c-294d-41b8-b106-1f6feb6364d5'} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:31:38.123412 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-3e61367c-294d-41b8-b106-1f6feb6364d5 admin admin] The token cache is being invalidate because project ef878af2a9184c309f244b3be501fd73 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80119) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 10:31:38.123887 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-3e61367c-294d-41b8-b106-1f6feb6364d5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:31:38.125842 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-3e61367c-294d-41b8-b106-1f6feb6364d5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:31:38.132308 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 876/1747] 10.222.0.49 () {58 vars in 1232 bytes} [Wed Jun 15 10:31:37 2022] DELETE /identity/v3/projects/ef878af2a9184c309f244b3be501fd73 => generated 0 bytes in 609 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:31:38.132308 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e61367c-294d-41b8-b106-1f6feb6364d5 admin admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:31:38.135524 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-3e61367c-294d-41b8-b106-1f6feb6364d5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:31:38.266789 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a090e4-1eb2-46ba-929e-aca210268690 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:31:38.267947 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a090e4-1eb2-46ba-929e-aca210268690 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:31:38.268334 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a090e4-1eb2-46ba-929e-aca210268690 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:31:38.268501 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a090e4-1eb2-46ba-929e-aca210268690 admin admin] PATH_INFO: `/v3/users/5a4b9994ac914cbcab64e37132baac9a` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:31:38.274064 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a090e4-1eb2-46ba-929e-aca210268690 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5a4b9994ac914cbcab64e37132baac9a)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:31:38.276880 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a090e4-1eb2-46ba-929e-aca210268690 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:31:38.348544 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e61367c-294d-41b8-b106-1f6feb6364d5 admin admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:31:38.370588 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e871e7b-4ea4-4e93-b455-969b4f1f8526 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:31:38.371302 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e871e7b-4ea4-4e93-b455-969b4f1f8526 admin admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:31:38.371562 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e871e7b-4ea4-4e93-b455-969b4f1f8526 admin admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:31:38.371973 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e871e7b-4ea4-4e93-b455-969b4f1f8526 admin admin] PATH_INFO: `/v3/domains` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:31:38.372843 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e871e7b-4ea4-4e93-b455-969b4f1f8526 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:31:38.373390 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e871e7b-4ea4-4e93-b455-969b4f1f8526 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:31:38.375124 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e871e7b-4ea4-4e93-b455-969b4f1f8526 admin admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:31:38.410468 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 877/1748] 10.222.0.49 () {58 vars in 1152 bytes} [Wed Jun 15 10:31:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:31:38.411658 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e871e7b-4ea4-4e93-b455-969b4f1f8526 admin admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:31:38.431020 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-7e871e7b-4ea4-4e93-b455-969b4f1f8526 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:31:38.483821 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-e0a090e4-1eb2-46ba-929e-aca210268690 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5a4b9994ac914cbcab64e37132baac9a'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:31:38.492548 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e871e7b-4ea4-4e93-b455-969b4f1f8526 admin admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:31:38.516000 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34447890-1cee-4202-b2c4-df56d1980aae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:31:38.517085 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34447890-1cee-4202-b2c4-df56d1980aae admin admin] REQUEST_METHOD: `DELETE` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:31:38.518091 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34447890-1cee-4202-b2c4-df56d1980aae admin admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:31:38.518524 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34447890-1cee-4202-b2c4-df56d1980aae admin admin] PATH_INFO: `/v3/users/39eef429feea40b9b743cebbbe22d680` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:31:38.522054 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34447890-1cee-4202-b2c4-df56d1980aae admin admin] RBAC: Authorizing `identity:delete_user(user_id=39eef429feea40b9b743cebbbe22d680)` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:31:38.523046 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-e0a090e4-1eb2-46ba-929e-aca210268690 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5a4b9994ac914cbcab64e37132baac9a'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:31:38.525603 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-e0a090e4-1eb2-46ba-929e-aca210268690 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5a4b9994ac914cbcab64e37132baac9a'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:31:38.526660 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-e0a090e4-1eb2-46ba-929e-aca210268690 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5a4b9994ac914cbcab64e37132baac9a'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:31:38.528055 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34447890-1cee-4202-b2c4-df56d1980aae admin admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:31:38.546147 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-e0a090e4-1eb2-46ba-929e-aca210268690 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5a4b9994ac914cbcab64e37132baac9a'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:31:38.642419 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 872/1749] 10.222.0.49 () {58 vars in 1223 bytes} [Wed Jun 15 10:31:37 2022] DELETE /identity/v3/users/5a4b9994ac914cbcab64e37132baac9a => generated 0 bytes in 670 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:31:38.643851 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a090e4-1eb2-46ba-929e-aca210268690 admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:31:38.646426 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-e0a090e4-1eb2-46ba-929e-aca210268690 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:31:38.728818 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-34447890-1cee-4202-b2c4-df56d1980aae admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '39eef429feea40b9b743cebbbe22d680'} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:31:38.762533 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-34447890-1cee-4202-b2c4-df56d1980aae admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '39eef429feea40b9b743cebbbe22d680'} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:31:38.764466 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-34447890-1cee-4202-b2c4-df56d1980aae admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '39eef429feea40b9b743cebbbe22d680'} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:31:38.768587 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-34447890-1cee-4202-b2c4-df56d1980aae admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '39eef429feea40b9b743cebbbe22d680'} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:31:38.794234 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-34447890-1cee-4202-b2c4-df56d1980aae admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '39eef429feea40b9b743cebbbe22d680'} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:31:38.839851 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 878/1750] 10.222.0.49 () {58 vars in 1223 bytes} [Wed Jun 15 10:31:38 2022] DELETE /identity/v3/users/39eef429feea40b9b743cebbbe22d680 => generated 0 bytes in 428 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:31:38.841632 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34447890-1cee-4202-b2c4-df56d1980aae admin admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:31:38.844139 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-34447890-1cee-4202-b2c4-df56d1980aae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:31:38.919753 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:31:38.957995 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a090e4-1eb2-46ba-929e-aca210268690 admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:31:39.030531 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34447890-1cee-4202-b2c4-df56d1980aae admin admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:31:39.046535 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2fcfecb-4b66-4b92-b495-4ccb7ccbf921 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:31:39.047893 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fcfecb-4b66-4b92-b495-4ccb7ccbf921 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:31:39.048398 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fcfecb-4b66-4b92-b495-4ccb7ccbf921 admin admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:31:39.049051 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fcfecb-4b66-4b92-b495-4ccb7ccbf921 admin admin] PATH_INFO: `/v3/projects/d56733aa8e8342e3b70961a9476f2005` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:31:39.049741 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:31:39.054293 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2fcfecb-4b66-4b92-b495-4ccb7ccbf921 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d56733aa8e8342e3b70961a9476f2005)` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:31:39.059450 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2fcfecb-4b66-4b92-b495-4ccb7ccbf921 admin admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:31:39.165543 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0d4e23-c91c-423a-a97b-68029c003dbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:31:39.165543 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0d4e23-c91c-423a-a97b-68029c003dbc admin admin] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:31:39.165543 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0d4e23-c91c-423a-a97b-68029c003dbc admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:31:39.166127 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0d4e23-c91c-423a-a97b-68029c003dbc admin admin] PATH_INFO: `/v3/projects` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:31:39.166421 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca0d4e23-c91c-423a-a97b-68029c003dbc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:31:39.168438 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca0d4e23-c91c-423a-a97b-68029c003dbc admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:31:39.195691 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-c2fcfecb-4b66-4b92-b495-4ccb7ccbf921 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd56733aa8e8342e3b70961a9476f2005', 'request_id': 'req-c2fcfecb-4b66-4b92-b495-4ccb7ccbf921'} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:31:39.216248 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-c2fcfecb-4b66-4b92-b495-4ccb7ccbf921 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd56733aa8e8342e3b70961a9476f2005', 'request_id': 'req-c2fcfecb-4b66-4b92-b495-4ccb7ccbf921'} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:31:39.251468 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-c2fcfecb-4b66-4b92-b495-4ccb7ccbf921 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd56733aa8e8342e3b70961a9476f2005', 'request_id': 'req-c2fcfecb-4b66-4b92-b495-4ccb7ccbf921'} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:31:39.262062 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 873/1751] 10.222.0.49 () {60 vars in 1152 bytes} [Wed Jun 15 10:31:38 2022] POST /identity/v3/projects => generated 405 bytes in 619 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 10:31:39.268391 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159aee9f-791f-4de1-9937-da0ef2ff197d None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:31:39.272920 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-c2fcfecb-4b66-4b92-b495-4ccb7ccbf921 admin admin] The token cache is being invalidate because project d56733aa8e8342e3b70961a9476f2005 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80119) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 10:31:39.273351 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-c2fcfecb-4b66-4b92-b495-4ccb7ccbf921 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:31:39.278117 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-c2fcfecb-4b66-4b92-b495-4ccb7ccbf921 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:31:39.279129 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159aee9f-791f-4de1-9937-da0ef2ff197d None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:31:39.283518 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159aee9f-791f-4de1-9937-da0ef2ff197d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:31:39.284206 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 879/1752] 10.222.0.49 () {58 vars in 1232 bytes} [Wed Jun 15 10:31:38 2022] DELETE /identity/v3/projects/d56733aa8e8342e3b70961a9476f2005 => generated 0 bytes in 440 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:31:39.285138 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2fcfecb-4b66-4b92-b495-4ccb7ccbf921 admin admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:31:39.287446 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-c2fcfecb-4b66-4b92-b495-4ccb7ccbf921 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:31:39.331124 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-159aee9f-791f-4de1-9937-da0ef2ff197d None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:31:39.333971 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-159aee9f-791f-4de1-9937-da0ef2ff197d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:31:39.451468 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2fcfecb-4b66-4b92-b495-4ccb7ccbf921 admin admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:31:39.528897 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 874/1753] 10.222.0.49 () {58 vars in 957 bytes} [Wed Jun 15 10:31:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 266 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:31:39.531647 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04aec07c-f452-42d0-81fc-b431e7e7c29d None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:31:39.532266 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04aec07c-f452-42d0-81fc-b431e7e7c29d None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:31:39.534084 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04aec07c-f452-42d0-81fc-b431e7e7c29d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:31:39.582535 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-04aec07c-f452-42d0-81fc-b431e7e7c29d None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:31:39.584303 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-04aec07c-f452-42d0-81fc-b431e7e7c29d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:31:39.608811 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9eb230-8068-4625-8f58-925729ad736f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:31:39.609837 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9eb230-8068-4625-8f58-925729ad736f admin admin] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:31:39.610130 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9eb230-8068-4625-8f58-925729ad736f admin admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:31:39.610412 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9eb230-8068-4625-8f58-925729ad736f admin admin] PATH_INFO: `/v3/users` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:31:39.613682 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9eb230-8068-4625-8f58-925729ad736f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:31:39.616193 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9eb230-8068-4625-8f58-925729ad736f admin admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:31:39.625686 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 875/1754] 10.222.0.49 () {58 vars in 957 bytes} [Wed Jun 15 10:31:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:31:39.641819 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:31:39.643376 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059d7d6b-e1b3-4072-9bf5-6fc9b4de9e7a None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:31:39.644219 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059d7d6b-e1b3-4072-9bf5-6fc9b4de9e7a None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:31:39.644759 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059d7d6b-e1b3-4072-9bf5-6fc9b4de9e7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:31:39.679305 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 880/1755] 10.222.0.49 () {60 vars in 1143 bytes} [Wed Jun 15 10:31:39 2022] POST /identity/v3/users => generated 303 bytes in 391 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 10:31:39.680679 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9eb230-8068-4625-8f58-925729ad736f admin admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:31:39.694634 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-fc9eb230-8068-4625-8f58-925729ad736f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:31:39.708073 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-059d7d6b-e1b3-4072-9bf5-6fc9b4de9e7a None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:31:39.709346 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-059d7d6b-e1b3-4072-9bf5-6fc9b4de9e7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:31:39.750106 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 876/1756] 10.222.0.49 () {58 vars in 957 bytes} [Wed Jun 15 10:31:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:31:39.752735 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059d7d6b-e1b3-4072-9bf5-6fc9b4de9e7a None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:31:39.782664 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059d7d6b-e1b3-4072-9bf5-6fc9b4de9e7a None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:31:39.794592 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6546c28-db1d-44ec-a1f6-bd2f2a5c208e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:31:39.796645 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6546c28-db1d-44ec-a1f6-bd2f2a5c208e admin admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:31:39.796966 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6546c28-db1d-44ec-a1f6-bd2f2a5c208e admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:31:39.797261 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6546c28-db1d-44ec-a1f6-bd2f2a5c208e admin admin] PATH_INFO: `/v3/roles` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:31:39.799103 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6546c28-db1d-44ec-a1f6-bd2f2a5c208e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:31:39.804319 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6546c28-db1d-44ec-a1f6-bd2f2a5c208e admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:31:39.837384 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 877/1757] 10.222.0.49 () {58 vars in 1121 bytes} [Wed Jun 15 10:31:39 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 10:31:39.840125 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6546c28-db1d-44ec-a1f6-bd2f2a5c208e admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:31:39.876955 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6546c28-db1d-44ec-a1f6-bd2f2a5c208e admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:31:39.893875 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e342eeb-9ff3-4354-84a8-53ab4430d97f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:31:39.896825 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e342eeb-9ff3-4354-84a8-53ab4430d97f admin admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:31:39.897194 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e342eeb-9ff3-4354-84a8-53ab4430d97f admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:31:39.897575 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9eb230-8068-4625-8f58-925729ad736f admin admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:31:39.897672 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e342eeb-9ff3-4354-84a8-53ab4430d97f admin admin] PATH_INFO: `/v3/domains` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:31:39.900833 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e342eeb-9ff3-4354-84a8-53ab4430d97f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:31:39.901162 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e342eeb-9ff3-4354-84a8-53ab4430d97f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:31:39.905404 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e342eeb-9ff3-4354-84a8-53ab4430d97f admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:31:39.912061 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f77ddf-76fa-4ce8-ab91-428e72a51cb5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0cb44a17d205431793e9a8791e477a32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '559b3e713f5844b6ae37151d56dd419f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:31:39.914297 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f77ddf-76fa-4ce8-ab91-428e72a51cb5 service neutron] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:31:39.914589 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f77ddf-76fa-4ce8-ab91-428e72a51cb5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:31:39.914898 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f77ddf-76fa-4ce8-ab91-428e72a51cb5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:31:39.950908 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f77ddf-76fa-4ce8-ab91-428e72a51cb5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:31:39.952440 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f77ddf-76fa-4ce8-ab91-428e72a51cb5 service neutron] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:31:39.953089 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 878/1758] 10.222.0.49 () {58 vars in 1152 bytes} [Wed Jun 15 10:31:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 116 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:31:39.955289 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e342eeb-9ff3-4354-84a8-53ab4430d97f admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:31:39.971355 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e342eeb-9ff3-4354-84a8-53ab4430d97f admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:31:39.977799 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77152c85-f15b-4506-bf5d-a8a531a3e88e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:31:39.978791 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77152c85-f15b-4506-bf5d-a8a531a3e88e admin admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:31:39.979099 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77152c85-f15b-4506-bf5d-a8a531a3e88e admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:31:39.979397 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77152c85-f15b-4506-bf5d-a8a531a3e88e admin admin] PATH_INFO: `/v3/roles` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:31:39.980354 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77152c85-f15b-4506-bf5d-a8a531a3e88e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:31:39.983010 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77152c85-f15b-4506-bf5d-a8a531a3e88e admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:31:39.998734 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 879/1759] 10.222.0.49 () {58 vars in 1121 bytes} [Wed Jun 15 10:31:39 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 10:31:39.998734 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77152c85-f15b-4506-bf5d-a8a531a3e88e admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:31:39.999893 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 881/1760] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Jun 15 10:31:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:31:40.002556 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f77ddf-76fa-4ce8-ab91-428e72a51cb5 service neutron] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:31:40.009074 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77152c85-f15b-4506-bf5d-a8a531a3e88e admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:31:40.014415 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8b75ee-5f00-4216-85ca-fc0fd6e42bbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:31:40.015122 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8b75ee-5f00-4216-85ca-fc0fd6e42bbe admin admin] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:31:40.015561 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8b75ee-5f00-4216-85ca-fc0fd6e42bbe admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:31:40.015883 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8b75ee-5f00-4216-85ca-fc0fd6e42bbe admin admin] PATH_INFO: `/v3/projects` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:31:40.016735 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b8b75ee-5f00-4216-85ca-fc0fd6e42bbe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:31:40.019607 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b8b75ee-5f00-4216-85ca-fc0fd6e42bbe admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:31:40.038808 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f77ddf-76fa-4ce8-ab91-428e72a51cb5 service neutron] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:31:40.048924 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981d48c1-cf6d-4a2e-a18d-bca71fa80965 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:31:40.049918 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981d48c1-cf6d-4a2e-a18d-bca71fa80965 admin admin] REQUEST_METHOD: `PUT` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:31:40.050205 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981d48c1-cf6d-4a2e-a18d-bca71fa80965 admin admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:31:40.050551 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981d48c1-cf6d-4a2e-a18d-bca71fa80965 admin admin] PATH_INFO: `/v3/projects/0e883186e4784ba4a94a7b7f649b6d43/users/318f2b2a3f024c7a8e56269feb343d9f/roles/6876e57cc51c448097ab109d35b2a3da` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:31:40.068434 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 880/1761] 10.222.0.49 () {60 vars in 1152 bytes} [Wed Jun 15 10:31:39 2022] POST /identity/v3/projects => generated 409 bytes in 72 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 10:31:40.073719 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8b75ee-5f00-4216-85ca-fc0fd6e42bbe admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:31:40.102937 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981d48c1-cf6d-4a2e-a18d-bca71fa80965 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0e883186e4784ba4a94a7b7f649b6d43, user_id=318f2b2a3f024c7a8e56269feb343d9f, role_id=6876e57cc51c448097ab109d35b2a3da)` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:31:40.106155 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981d48c1-cf6d-4a2e-a18d-bca71fa80965 admin admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:31:40.159205 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 882/1762] 10.222.0.49 () {60 vars in 1482 bytes} [Wed Jun 15 10:31:39 2022] PUT /identity/v3/projects/0e883186e4784ba4a94a7b7f649b6d43/users/318f2b2a3f024c7a8e56269feb343d9f/roles/6876e57cc51c448097ab109d35b2a3da => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:31:40.166912 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a66d8e-97df-4b5a-ae86-9936ce95da79 None None] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:31:40.169701 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a66d8e-97df-4b5a-ae86-9936ce95da79 None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:31:40.170747 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a66d8e-97df-4b5a-ae86-9936ce95da79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:31:40.206122 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:31:40.226722 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.auth.core [None req-a8a66d8e-97df-4b5a-ae86-9936ce95da79 None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:31:40.228608 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-a8a66d8e-97df-4b5a-ae86-9936ce95da79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:31:40.249479 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8b75ee-5f00-4216-85ca-fc0fd6e42bbe admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:31:40.389968 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0fba30f-fcb2-4592-a316-9686c2458f54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:31:40.390612 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fba30f-fcb2-4592-a316-9686c2458f54 admin admin] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:31:40.390991 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fba30f-fcb2-4592-a316-9686c2458f54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:31:40.391364 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fba30f-fcb2-4592-a316-9686c2458f54 admin admin] PATH_INFO: `/v3/users` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:31:40.392145 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0fba30f-fcb2-4592-a316-9686c2458f54 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:31:40.397081 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0fba30f-fcb2-4592-a316-9686c2458f54 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:31:40.418662 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 883/1763] 10.222.0.49 () {58 vars in 957 bytes} [Wed Jun 15 10:31:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 255 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:31:40.440911 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 881/1764] 10.222.0.49 () {60 vars in 1143 bytes} [Wed Jun 15 10:31:40 2022] POST /identity/v3/users => generated 305 bytes in 368 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 10:31:40.445032 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a66d8e-97df-4b5a-ae86-9936ce95da79 None None] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:31:40.461001 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a66d8e-97df-4b5a-ae86-9936ce95da79 None None] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:31:40.475585 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84a7c31-3146-4a4b-97db-b72d618d390a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:31:40.476458 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84a7c31-3146-4a4b-97db-b72d618d390a admin admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:31:40.478507 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84a7c31-3146-4a4b-97db-b72d618d390a admin admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:31:40.479052 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84a7c31-3146-4a4b-97db-b72d618d390a admin admin] PATH_INFO: `/v3/roles` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:31:40.481131 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d84a7c31-3146-4a4b-97db-b72d618d390a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:31:40.486380 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d84a7c31-3146-4a4b-97db-b72d618d390a admin admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:31:40.515780 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 884/1765] 10.222.0.49 () {58 vars in 1121 bytes} [Wed Jun 15 10:31:40 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 10:31:40.519872 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0fba30f-fcb2-4592-a316-9686c2458f54 admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:31:40.556421 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0fba30f-fcb2-4592-a316-9686c2458f54 admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:31:40.565954 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a134289-135d-45be-9821-20050a4a79ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:31:40.566965 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a134289-135d-45be-9821-20050a4a79ed admin admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:31:40.567503 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a134289-135d-45be-9821-20050a4a79ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:31:40.568402 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a134289-135d-45be-9821-20050a4a79ed admin admin] PATH_INFO: `/v3/roles` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:31:40.569368 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a134289-135d-45be-9821-20050a4a79ed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:31:40.571278 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a134289-135d-45be-9821-20050a4a79ed admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:31:40.593553 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 882/1766] 10.222.0.49 () {58 vars in 1121 bytes} [Wed Jun 15 10:31:40 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 10:31:40.598199 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84a7c31-3146-4a4b-97db-b72d618d390a admin admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:31:40.622473 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84a7c31-3146-4a4b-97db-b72d618d390a admin admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:31:40.652652 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fabe21d9-de9e-4aab-b5cb-b2b8ce803cca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:31:40.655724 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabe21d9-de9e-4aab-b5cb-b2b8ce803cca admin admin] REQUEST_METHOD: `PUT` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:31:40.656439 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabe21d9-de9e-4aab-b5cb-b2b8ce803cca admin admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:31:40.656789 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabe21d9-de9e-4aab-b5cb-b2b8ce803cca admin admin] PATH_INFO: `/v3/projects/38a3de944c244ae4a8b80f68bda69515/users/68ab1e11148e413386dd8aa4aefae2ce/roles/6876e57cc51c448097ab109d35b2a3da` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:31:40.691399 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fabe21d9-de9e-4aab-b5cb-b2b8ce803cca admin admin] RBAC: Authorizing `identity:create_grant(project_id=38a3de944c244ae4a8b80f68bda69515, user_id=68ab1e11148e413386dd8aa4aefae2ce, role_id=6876e57cc51c448097ab109d35b2a3da)` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:31:40.694674 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fabe21d9-de9e-4aab-b5cb-b2b8ce803cca admin admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:31:40.729253 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a134289-135d-45be-9821-20050a4a79ed admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:31:40.730248 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 885/1767] 10.222.0.49 () {60 vars in 1482 bytes} [Wed Jun 15 10:31:40 2022] PUT /identity/v3/projects/38a3de944c244ae4a8b80f68bda69515/users/68ab1e11148e413386dd8aa4aefae2ce/roles/6876e57cc51c448097ab109d35b2a3da => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:31:40.746008 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9a4cef-1f60-46fb-9070-523ec02f92e7 None None] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:31:40.746351 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9a4cef-1f60-46fb-9070-523ec02f92e7 None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:31:40.746811 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9a4cef-1f60-46fb-9070-523ec02f92e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:31:40.858969 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.auth.core [None req-ef9a4cef-1f60-46fb-9070-523ec02f92e7 None None] MFA Rules not processed for user `68ab1e11148e413386dd8aa4aefae2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:31:40.868987 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-ef9a4cef-1f60-46fb-9070-523ec02f92e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:31:40.909891 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a134289-135d-45be-9821-20050a4a79ed admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:31:40.951903 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:INSERT=5,keystone:SELECT=95 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:31:41.026649 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780f1494-2004-48f7-aa91-1c4422191327 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0cb44a17d205431793e9a8791e477a32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '559b3e713f5844b6ae37151d56dd419f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:31:41.027296 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780f1494-2004-48f7-aa91-1c4422191327 service neutron] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:31:41.027648 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780f1494-2004-48f7-aa91-1c4422191327 service neutron] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:31:41.028470 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780f1494-2004-48f7-aa91-1c4422191327 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:31:41.108766 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 886/1768] 10.222.0.49 () {58 vars in 957 bytes} [Wed Jun 15 10:31:40 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 364 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 10:31:41.121273 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9a4cef-1f60-46fb-9070-523ec02f92e7 None None] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:31:41.123982 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:31:41.177272 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780f1494-2004-48f7-aa91-1c4422191327 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:31:41.178852 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780f1494-2004-48f7-aa91-1c4422191327 service neutron] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:31:41.230786 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 883/1769] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Jun 15 10:31:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 502 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 10:31:41.325990 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9a4cef-1f60-46fb-9070-523ec02f92e7 None None] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:31:41.341794 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d19e1c-f7fb-4a68-8703-a636b3d1c1fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:31:41.342738 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d19e1c-f7fb-4a68-8703-a636b3d1c1fd admin admin] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:31:41.343033 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d19e1c-f7fb-4a68-8703-a636b3d1c1fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:31:41.344749 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d19e1c-f7fb-4a68-8703-a636b3d1c1fd admin admin] PATH_INFO: `/v3/projects` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:31:41.346054 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d19e1c-f7fb-4a68-8703-a636b3d1c1fd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:31:41.359287 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d19e1c-f7fb-4a68-8703-a636b3d1c1fd admin admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:31:41.452772 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 887/1770] 10.222.0.49 () {60 vars in 1152 bytes} [Wed Jun 15 10:31:41 2022] POST /identity/v3/projects => generated 407 bytes in 334 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 10:31:41.455628 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780f1494-2004-48f7-aa91-1c4422191327 service neutron] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:31:41.477787 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780f1494-2004-48f7-aa91-1c4422191327 service neutron] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:31:41.683177 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9dad436-4006-4963-ba5d-c02bfc12f670 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:31:41.683889 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9dad436-4006-4963-ba5d-c02bfc12f670 admin admin] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:31:41.684183 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9dad436-4006-4963-ba5d-c02bfc12f670 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:31:41.684468 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9dad436-4006-4963-ba5d-c02bfc12f670 admin admin] PATH_INFO: `/v3/users` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:31:41.685190 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9dad436-4006-4963-ba5d-c02bfc12f670 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:31:41.687198 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9dad436-4006-4963-ba5d-c02bfc12f670 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:31:41.724815 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 884/1771] 10.222.0.49 () {60 vars in 1143 bytes} [Wed Jun 15 10:31:41 2022] POST /identity/v3/users => generated 310 bytes in 267 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 10:31:41.726446 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d19e1c-f7fb-4a68-8703-a636b3d1c1fd admin admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:31:41.770481 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d19e1c-f7fb-4a68-8703-a636b3d1c1fd admin admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:31:41.798869 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a2038e-f36e-4f9a-a0dd-1c7619213917 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:31:41.800480 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a2038e-f36e-4f9a-a0dd-1c7619213917 admin admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:31:41.800799 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a2038e-f36e-4f9a-a0dd-1c7619213917 admin admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:31:41.803222 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a2038e-f36e-4f9a-a0dd-1c7619213917 admin admin] PATH_INFO: `/v3/roles` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:31:41.804308 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a2038e-f36e-4f9a-a0dd-1c7619213917 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:31:41.806558 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a2038e-f36e-4f9a-a0dd-1c7619213917 admin admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:31:41.858004 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 888/1772] 10.222.0.49 () {58 vars in 1121 bytes} [Wed Jun 15 10:31:41 2022] GET /identity/v3/roles => generated 1420 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 10:31:41.860404 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9dad436-4006-4963-ba5d-c02bfc12f670 admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:31:41.876294 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9dad436-4006-4963-ba5d-c02bfc12f670 admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:31:41.885217 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72455f41-4fd6-4716-95be-8bb2ced870e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:31:41.886259 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72455f41-4fd6-4716-95be-8bb2ced870e6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:31:41.886692 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72455f41-4fd6-4716-95be-8bb2ced870e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:31:41.888653 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72455f41-4fd6-4716-95be-8bb2ced870e6 admin admin] PATH_INFO: `/v3/projects/38ea0331c1d847c09dc1078372d6cba6/users/128adcb269f544a5bb7c2f5a232c6b08/roles/8051174e7a7947d5b991ebd793c2e543` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:31:41.936178 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72455f41-4fd6-4716-95be-8bb2ced870e6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=38ea0331c1d847c09dc1078372d6cba6, user_id=128adcb269f544a5bb7c2f5a232c6b08, role_id=8051174e7a7947d5b991ebd793c2e543)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:31:41.938931 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72455f41-4fd6-4716-95be-8bb2ced870e6 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:31:41.972750 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 885/1773] 10.222.0.49 () {60 vars in 1482 bytes} [Wed Jun 15 10:31:41 2022] PUT /identity/v3/projects/38ea0331c1d847c09dc1078372d6cba6/users/128adcb269f544a5bb7c2f5a232c6b08/roles/8051174e7a7947d5b991ebd793c2e543 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:31:41.976696 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a2038e-f36e-4f9a-a0dd-1c7619213917 admin admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:31:41.997408 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a2038e-f36e-4f9a-a0dd-1c7619213917 admin admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:31:42.141883 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f8e183-3efc-40cd-8334-bb39c690eded admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:31:42.142525 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f8e183-3efc-40cd-8334-bb39c690eded admin admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:31:42.142835 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f8e183-3efc-40cd-8334-bb39c690eded admin admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:31:42.143156 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f8e183-3efc-40cd-8334-bb39c690eded admin admin] PATH_INFO: `/v3/roles` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:31:42.143929 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f8e183-3efc-40cd-8334-bb39c690eded admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:31:42.145591 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f8e183-3efc-40cd-8334-bb39c690eded admin admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:31:42.170981 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 889/1774] 10.222.0.49 () {58 vars in 1121 bytes} [Wed Jun 15 10:31:41 2022] GET /identity/v3/roles => generated 1420 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 10:31:42.174729 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72455f41-4fd6-4716-95be-8bb2ced870e6 admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:31:42.188005 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72455f41-4fd6-4716-95be-8bb2ced870e6 admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:31:42.195120 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54cf9530-04ea-488c-9c3f-a136b2bb833f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:31:42.196587 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cf9530-04ea-488c-9c3f-a136b2bb833f admin admin] REQUEST_METHOD: `PUT` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:31:42.197100 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cf9530-04ea-488c-9c3f-a136b2bb833f admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:31:42.197499 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cf9530-04ea-488c-9c3f-a136b2bb833f admin admin] PATH_INFO: `/v3/projects/38ea0331c1d847c09dc1078372d6cba6/users/128adcb269f544a5bb7c2f5a232c6b08/roles/8051174e7a7947d5b991ebd793c2e543` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:31:42.200350 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54cf9530-04ea-488c-9c3f-a136b2bb833f admin admin] RBAC: Authorizing `identity:create_grant(project_id=38ea0331c1d847c09dc1078372d6cba6, user_id=128adcb269f544a5bb7c2f5a232c6b08, role_id=8051174e7a7947d5b991ebd793c2e543)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:31:42.203620 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54cf9530-04ea-488c-9c3f-a136b2bb833f admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:31:42.218005 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 886/1775] 10.222.0.49 () {60 vars in 1482 bytes} [Wed Jun 15 10:31:42 2022] PUT /identity/v3/projects/38ea0331c1d847c09dc1078372d6cba6/users/128adcb269f544a5bb7c2f5a232c6b08/roles/8051174e7a7947d5b991ebd793c2e543 => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:31:42.230111 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e5d2e3-cf1f-49d5-867a-254f70918dd9 None None] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:31:42.230818 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e5d2e3-cf1f-49d5-867a-254f70918dd9 None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:31:42.231372 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e5d2e3-cf1f-49d5-867a-254f70918dd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:31:42.311025 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.auth.core [None req-66e5d2e3-cf1f-49d5-867a-254f70918dd9 None None] MFA Rules not processed for user `128adcb269f544a5bb7c2f5a232c6b08`. Rule list: `[]` (Enabled: `True`). {{(pid=80119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:31:42.312917 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-66e5d2e3-cf1f-49d5-867a-254f70918dd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:31:42.361610 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:31:42.481336 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 890/1776] 10.222.0.49 () {58 vars in 957 bytes} [Wed Jun 15 10:31:42 2022] POST /identity/v3/auth/tokens => generated 3341 bytes in 252 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 10:31:42.491714 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54cf9530-04ea-488c-9c3f-a136b2bb833f admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:31:42.494260 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-54cf9530-04ea-488c-9c3f-a136b2bb833f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:31:42.542857 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e5d2e3-cf1f-49d5-867a-254f70918dd9 None None] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:31:42.544951 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-66e5d2e3-cf1f-49d5-867a-254f70918dd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:31:42.687050 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54cf9530-04ea-488c-9c3f-a136b2bb833f admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:31:42.691925 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fdf5bb4-6c25-4b60-9031-a8af7d6c3033 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20d1327553404b80aecee87c78fcb52b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '559b3e713f5844b6ae37151d56dd419f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:31:42.692996 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdf5bb4-6c25-4b60-9031-a8af7d6c3033 service cinder] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:31:42.693280 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdf5bb4-6c25-4b60-9031-a8af7d6c3033 service cinder] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:31:42.693552 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdf5bb4-6c25-4b60-9031-a8af7d6c3033 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:31:42.705560 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fdf5bb4-6c25-4b60-9031-a8af7d6c3033 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:31:42.707082 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fdf5bb4-6c25-4b60-9031-a8af7d6c3033 service cinder] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:31:42.731798 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e5d2e3-cf1f-49d5-867a-254f70918dd9 None None] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:31:42.735253 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 887/1777] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Jun 15 10:31:42 2022] GET /identity/v3/auth/tokens => generated 3341 bytes in 245 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:31:42.737720 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fdf5bb4-6c25-4b60-9031-a8af7d6c3033 service cinder] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:31:42.743195 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-1fdf5bb4-6c25-4b60-9031-a8af7d6c3033 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:31:42.744063 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5799047-02f5-4ebb-95f4-69258de7732a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:31:42.746976 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5799047-02f5-4ebb-95f4-69258de7732a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:31:42.747365 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5799047-02f5-4ebb-95f4-69258de7732a admin admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:31:42.750440 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5799047-02f5-4ebb-95f4-69258de7732a admin admin] PATH_INFO: `/v3/projects/abb687a167db47dbb74b2654087c2cac` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:31:42.752501 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5799047-02f5-4ebb-95f4-69258de7732a admin admin] RBAC: Authorizing `identity:delete_project(project_id=abb687a167db47dbb74b2654087c2cac)` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:31:42.758232 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5799047-02f5-4ebb-95f4-69258de7732a admin admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:31:42.784550 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fdf5bb4-6c25-4b60-9031-a8af7d6c3033 service cinder] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:31:42.803497 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4a3a9b-34a1-4d11-87cc-0028e0f739f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:31:42.804382 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4a3a9b-34a1-4d11-87cc-0028e0f739f8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:31:42.804661 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4a3a9b-34a1-4d11-87cc-0028e0f739f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:31:42.804933 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4a3a9b-34a1-4d11-87cc-0028e0f739f8 admin admin] PATH_INFO: `/v3/projects/6a7f760928d048fd8d39c90c7796515d` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:31:42.810672 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be4a3a9b-34a1-4d11-87cc-0028e0f739f8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6a7f760928d048fd8d39c90c7796515d)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:31:42.812636 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be4a3a9b-34a1-4d11-87cc-0028e0f739f8 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:31:42.867783 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-f5799047-02f5-4ebb-95f4-69258de7732a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'abb687a167db47dbb74b2654087c2cac', 'request_id': 'req-f5799047-02f5-4ebb-95f4-69258de7732a'} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:31:42.886220 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-f5799047-02f5-4ebb-95f4-69258de7732a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'abb687a167db47dbb74b2654087c2cac', 'request_id': 'req-f5799047-02f5-4ebb-95f4-69258de7732a'} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:31:42.888263 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=94,keystone:INSERT=5,keystone:DELETE=1 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:31:42.895135 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-f5799047-02f5-4ebb-95f4-69258de7732a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'abb687a167db47dbb74b2654087c2cac', 'request_id': 'req-f5799047-02f5-4ebb-95f4-69258de7732a'} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:31:42.922273 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-f5799047-02f5-4ebb-95f4-69258de7732a admin admin] The token cache is being invalidate because project abb687a167db47dbb74b2654087c2cac was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80119) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 10:31:42.922589 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-f5799047-02f5-4ebb-95f4-69258de7732a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:31:42.924188 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-f5799047-02f5-4ebb-95f4-69258de7732a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:31:42.929156 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 891/1778] 10.222.0.49 () {58 vars in 1232 bytes} [Wed Jun 15 10:31:42 2022] DELETE /identity/v3/projects/abb687a167db47dbb74b2654087c2cac => generated 0 bytes in 387 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 10:31:42.934558 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33c97eb-ff74-4ea3-b5f7-a05af91c150b None None] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:31:42.934901 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33c97eb-ff74-4ea3-b5f7-a05af91c150b None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:31:42.935398 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33c97eb-ff74-4ea3-b5f7-a05af91c150b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:31:42.949109 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-be4a3a9b-34a1-4d11-87cc-0028e0f739f8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6a7f760928d048fd8d39c90c7796515d', 'request_id': 'req-be4a3a9b-34a1-4d11-87cc-0028e0f739f8'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:31:42.970402 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-be4a3a9b-34a1-4d11-87cc-0028e0f739f8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6a7f760928d048fd8d39c90c7796515d', 'request_id': 'req-be4a3a9b-34a1-4d11-87cc-0028e0f739f8'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:31:42.981888 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-be4a3a9b-34a1-4d11-87cc-0028e0f739f8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6a7f760928d048fd8d39c90c7796515d', 'request_id': 'req-be4a3a9b-34a1-4d11-87cc-0028e0f739f8'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:31:42.991717 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.auth.core [None req-f33c97eb-ff74-4ea3-b5f7-a05af91c150b None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:31:42.994515 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-f33c97eb-ff74-4ea3-b5f7-a05af91c150b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:31:43.006294 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-be4a3a9b-34a1-4d11-87cc-0028e0f739f8 admin admin] The token cache is being invalidate because project 6a7f760928d048fd8d39c90c7796515d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80120) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 10:31:43.006652 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-be4a3a9b-34a1-4d11-87cc-0028e0f739f8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:31:43.013808 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-be4a3a9b-34a1-4d11-87cc-0028e0f739f8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:31:43.023390 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 888/1779] 10.222.0.49 () {58 vars in 1232 bytes} [Wed Jun 15 10:31:42 2022] DELETE /identity/v3/projects/6a7f760928d048fd8d39c90c7796515d => generated 0 bytes in 284 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:31:43.026698 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd326a03-8713-4f5a-bff2-c692ab18d85c None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:31:43.027387 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd326a03-8713-4f5a-bff2-c692ab18d85c None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:31:43.027514 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd326a03-8713-4f5a-bff2-c692ab18d85c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:31:43.061108 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-dd326a03-8713-4f5a-bff2-c692ab18d85c None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:31:43.062035 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-dd326a03-8713-4f5a-bff2-c692ab18d85c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:31:43.121763 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:31:43.166690 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 892/1780] 10.222.0.49 () {58 vars in 957 bytes} [Wed Jun 15 10:31:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 10:31:43.168653 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33c97eb-ff74-4ea3-b5f7-a05af91c150b None None] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:31:43.173017 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-f33c97eb-ff74-4ea3-b5f7-a05af91c150b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:31:43.244960 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 889/1781] 10.222.0.49 () {58 vars in 957 bytes} [Wed Jun 15 10:31:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 10:31:43.246595 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30baef86-a6cb-445b-9a15-0f3c145ed6c0 None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:31:43.246890 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30baef86-a6cb-445b-9a15-0f3c145ed6c0 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:31:43.248843 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30baef86-a6cb-445b-9a15-0f3c145ed6c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:31:43.290183 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-30baef86-a6cb-445b-9a15-0f3c145ed6c0 None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:31:43.292324 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-30baef86-a6cb-445b-9a15-0f3c145ed6c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:31:43.311936 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 890/1782] 10.222.0.49 () {58 vars in 957 bytes} [Wed Jun 15 10:31:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 66 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:31:43.316861 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940ba115-9e33-4ee3-8b59-e309220b20f1 None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:31:43.317169 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940ba115-9e33-4ee3-8b59-e309220b20f1 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:31:43.317452 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940ba115-9e33-4ee3-8b59-e309220b20f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:31:43.344544 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-940ba115-9e33-4ee3-8b59-e309220b20f1 None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:31:43.346131 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-940ba115-9e33-4ee3-8b59-e309220b20f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:31:43.349201 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33c97eb-ff74-4ea3-b5f7-a05af91c150b None None] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:31:43.353283 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fac1ee1-1bf3-4e0e-bde0-bbeccda01730 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20d1327553404b80aecee87c78fcb52b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '559b3e713f5844b6ae37151d56dd419f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:31:43.353959 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fac1ee1-1bf3-4e0e-bde0-bbeccda01730 service cinder] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:31:43.354336 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fac1ee1-1bf3-4e0e-bde0-bbeccda01730 service cinder] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:31:43.354829 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fac1ee1-1bf3-4e0e-bde0-bbeccda01730 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:31:43.356467 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-5fac1ee1-1bf3-4e0e-bde0-bbeccda01730 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:31:43.363415 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 891/1783] 10.222.0.49 () {58 vars in 957 bytes} [Wed Jun 15 10:31:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:31:43.364708 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940ba115-9e33-4ee3-8b59-e309220b20f1 None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:31:43.392976 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940ba115-9e33-4ee3-8b59-e309220b20f1 None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:31:43.409542 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b80c3d-d905-4e06-9a9b-b3ea8b8c13b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:31:43.410247 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b80c3d-d905-4e06-9a9b-b3ea8b8c13b3 admin admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:31:43.410685 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b80c3d-d905-4e06-9a9b-b3ea8b8c13b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:31:43.410972 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b80c3d-d905-4e06-9a9b-b3ea8b8c13b3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:31:43.411747 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b80c3d-d905-4e06-9a9b-b3ea8b8c13b3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:31:43.412061 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b80c3d-d905-4e06-9a9b-b3ea8b8c13b3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:31:43.416712 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b80c3d-d905-4e06-9a9b-b3ea8b8c13b3 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:31:43.447569 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 892/1784] 10.222.0.49 () {58 vars in 1152 bytes} [Wed Jun 15 10:31:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:31:43.456010 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b80c3d-d905-4e06-9a9b-b3ea8b8c13b3 admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:31:43.464683 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fac1ee1-1bf3-4e0e-bde0-bbeccda01730 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:31:43.470426 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fac1ee1-1bf3-4e0e-bde0-bbeccda01730 service cinder] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:31:43.474719 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b80c3d-d905-4e06-9a9b-b3ea8b8c13b3 admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:31:43.483024 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a804cb-a0da-4159-912a-cefb695ac083 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:31:43.492726 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a804cb-a0da-4159-912a-cefb695ac083 admin admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:31:43.495635 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a804cb-a0da-4159-912a-cefb695ac083 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:31:43.496215 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a804cb-a0da-4159-912a-cefb695ac083 admin admin] PATH_INFO: `/v3/domains` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:31:43.497375 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a804cb-a0da-4159-912a-cefb695ac083 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:31:43.497952 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a804cb-a0da-4159-912a-cefb695ac083 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:31:43.501914 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a804cb-a0da-4159-912a-cefb695ac083 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:31:43.508175 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 893/1785] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Jun 15 10:31:43 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:31:43.509231 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fac1ee1-1bf3-4e0e-bde0-bbeccda01730 service cinder] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:31:43.536710 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 893/1786] 10.222.0.49 () {58 vars in 1152 bytes} [Wed Jun 15 10:31:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:31:43.540138 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a804cb-a0da-4159-912a-cefb695ac083 admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:31:43.552174 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a804cb-a0da-4159-912a-cefb695ac083 admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:31:43.554078 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fac1ee1-1bf3-4e0e-bde0-bbeccda01730 service cinder] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:31:43.559674 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c52048-5f9c-44d5-b5ea-00853bef1b8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:31:43.560645 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c52048-5f9c-44d5-b5ea-00853bef1b8a admin admin] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:31:43.561300 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c52048-5f9c-44d5-b5ea-00853bef1b8a admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:31:43.562267 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c52048-5f9c-44d5-b5ea-00853bef1b8a admin admin] PATH_INFO: `/v3/projects` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:31:43.563288 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c52048-5f9c-44d5-b5ea-00853bef1b8a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:31:43.565813 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c52048-5f9c-44d5-b5ea-00853bef1b8a admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:31:43.574501 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26135bf4-9645-40b1-9104-76319dd6786b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:31:43.578689 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26135bf4-9645-40b1-9104-76319dd6786b admin admin] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:31:43.578996 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26135bf4-9645-40b1-9104-76319dd6786b admin admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:31:43.579300 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26135bf4-9645-40b1-9104-76319dd6786b admin admin] PATH_INFO: `/v3/projects` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:31:43.584195 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26135bf4-9645-40b1-9104-76319dd6786b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:31:43.592623 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26135bf4-9645-40b1-9104-76319dd6786b admin admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:31:43.617313 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 894/1787] 10.222.0.49 () {60 vars in 1152 bytes} [Wed Jun 15 10:31:43 2022] POST /identity/v3/projects => generated 405 bytes in 79 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 10:31:43.619549 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c52048-5f9c-44d5-b5ea-00853bef1b8a admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:31:43.679498 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 894/1788] 10.222.0.49 () {60 vars in 1152 bytes} [Wed Jun 15 10:31:43 2022] POST /identity/v3/projects => generated 367 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 10:31:43.683383 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26135bf4-9645-40b1-9104-76319dd6786b admin admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:31:43.796402 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=92,keystone:DELETE=6,keystone:INSERT=2 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:31:43.828683 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c52048-5f9c-44d5-b5ea-00853bef1b8a admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:31:43.885397 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26135bf4-9645-40b1-9104-76319dd6786b admin admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:31:43.901098 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c8b2b4-9b37-468c-a0cd-2c22dee325f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:31:43.902362 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c8b2b4-9b37-468c-a0cd-2c22dee325f9 admin admin] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:31:43.903930 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c8b2b4-9b37-468c-a0cd-2c22dee325f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:31:43.904446 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c8b2b4-9b37-468c-a0cd-2c22dee325f9 admin admin] PATH_INFO: `/v3/users` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:31:43.907209 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c8b2b4-9b37-468c-a0cd-2c22dee325f9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:31:43.913583 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c8b2b4-9b37-468c-a0cd-2c22dee325f9 admin admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:31:43.951209 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce026ec6-dbed-4c97-a9d2-6a0bb9e6294d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:31:43.951939 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce026ec6-dbed-4c97-a9d2-6a0bb9e6294d admin admin] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:31:43.952231 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce026ec6-dbed-4c97-a9d2-6a0bb9e6294d admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:31:43.952511 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce026ec6-dbed-4c97-a9d2-6a0bb9e6294d admin admin] PATH_INFO: `/v3/users` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:31:43.953248 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce026ec6-dbed-4c97-a9d2-6a0bb9e6294d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:31:43.955274 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce026ec6-dbed-4c97-a9d2-6a0bb9e6294d admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:31:43.989821 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 895/1789] 10.222.0.49 () {60 vars in 1143 bytes} [Wed Jun 15 10:31:43 2022] POST /identity/v3/users => generated 284 bytes in 307 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 10:31:43.995328 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c8b2b4-9b37-468c-a0cd-2c22dee325f9 admin admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:31:44.013760 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 895/1790] 10.222.0.49 () {60 vars in 1143 bytes} [Wed Jun 15 10:31:43 2022] POST /identity/v3/users => generated 303 bytes in 393 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 10:31:44.015507 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce026ec6-dbed-4c97-a9d2-6a0bb9e6294d admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:31:44.018164 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c8b2b4-9b37-468c-a0cd-2c22dee325f9 admin admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:31:44.030494 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ead60e-6b0c-4911-958d-ea7eb5a07934 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:31:44.031270 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ead60e-6b0c-4911-958d-ea7eb5a07934 admin admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:31:44.032178 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ead60e-6b0c-4911-958d-ea7eb5a07934 admin admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:31:44.032562 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ead60e-6b0c-4911-958d-ea7eb5a07934 admin admin] PATH_INFO: `/v3/roles` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:31:44.034459 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ead60e-6b0c-4911-958d-ea7eb5a07934 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:31:44.040681 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce026ec6-dbed-4c97-a9d2-6a0bb9e6294d admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:31:44.041741 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ead60e-6b0c-4911-958d-ea7eb5a07934 admin admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:31:44.054936 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:31:44.055116 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e8c345-154d-4275-b818-30fd5789e274 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:31:44.056696 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e8c345-154d-4275-b818-30fd5789e274 admin admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:31:44.057298 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e8c345-154d-4275-b818-30fd5789e274 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:31:44.057753 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e8c345-154d-4275-b818-30fd5789e274 admin admin] PATH_INFO: `/v3/roles` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:31:44.059486 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e8c345-154d-4275-b818-30fd5789e274 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:31:44.061149 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e8c345-154d-4275-b818-30fd5789e274 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:31:44.079584 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 896/1791] 10.222.0.49 () {58 vars in 1121 bytes} [Wed Jun 15 10:31:43 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 10:31:44.080894 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ead60e-6b0c-4911-958d-ea7eb5a07934 admin admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:31:44.105154 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 896/1792] 10.222.0.49 () {58 vars in 1121 bytes} [Wed Jun 15 10:31:44 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 15 10:31:44.107057 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e8c345-154d-4275-b818-30fd5789e274 admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:31:44.114200 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ead60e-6b0c-4911-958d-ea7eb5a07934 admin admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:31:44.128491 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea8234f-5d54-4c2f-b196-8c64a27d4833 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:31:44.130571 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea8234f-5d54-4c2f-b196-8c64a27d4833 admin admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:31:44.130571 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea8234f-5d54-4c2f-b196-8c64a27d4833 admin admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:31:44.130950 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea8234f-5d54-4c2f-b196-8c64a27d4833 admin admin] PATH_INFO: `/v3/roles` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:31:44.132284 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea8234f-5d54-4c2f-b196-8c64a27d4833 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:31:44.134201 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea8234f-5d54-4c2f-b196-8c64a27d4833 admin admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:31:44.137213 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e8c345-154d-4275-b818-30fd5789e274 admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:31:44.148028 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf34a17-5af0-45ca-b92a-262e46c516ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:31:44.148973 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf34a17-5af0-45ca-b92a-262e46c516ca admin admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:31:44.150068 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf34a17-5af0-45ca-b92a-262e46c516ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:31:44.150508 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf34a17-5af0-45ca-b92a-262e46c516ca admin admin] PATH_INFO: `/v3/roles` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:31:44.151433 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf34a17-5af0-45ca-b92a-262e46c516ca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:31:44.153321 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf34a17-5af0-45ca-b92a-262e46c516ca admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:31:44.168086 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 897/1793] 10.222.0.49 () {58 vars in 1121 bytes} [Wed Jun 15 10:31:44 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 10:31:44.169102 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea8234f-5d54-4c2f-b196-8c64a27d4833 admin admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:31:44.178801 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 897/1794] 10.222.0.49 () {58 vars in 1121 bytes} [Wed Jun 15 10:31:44 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 10:31:44.182683 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf34a17-5af0-45ca-b92a-262e46c516ca admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:31:44.202169 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf34a17-5af0-45ca-b92a-262e46c516ca admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:31:44.209262 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea8234f-5d54-4c2f-b196-8c64a27d4833 admin admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:31:44.215907 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee574e2-fb06-427f-b8ff-a99c859518bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:31:44.216833 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee574e2-fb06-427f-b8ff-a99c859518bb admin admin] REQUEST_METHOD: `PUT` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:31:44.217147 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee574e2-fb06-427f-b8ff-a99c859518bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:31:44.217432 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee574e2-fb06-427f-b8ff-a99c859518bb admin admin] PATH_INFO: `/v3/projects/8586f82366634d49a9155468fd73e451/users/875072bd53cc4b4d9842f8a89a47fdc8/roles/6876e57cc51c448097ab109d35b2a3da` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:31:44.223954 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329126f3-ce2d-450c-a108-f9c0229abc8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:31:44.225266 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329126f3-ce2d-450c-a108-f9c0229abc8b admin admin] REQUEST_METHOD: `PUT` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:31:44.225676 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329126f3-ce2d-450c-a108-f9c0229abc8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:31:44.226082 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329126f3-ce2d-450c-a108-f9c0229abc8b admin admin] PATH_INFO: `/v3/projects/c4d17fc4571545cebec33d03b677a6c1/users/4798b64da9a54711a2edf6679fd4f32e/roles/6876e57cc51c448097ab109d35b2a3da` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:31:44.248890 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee574e2-fb06-427f-b8ff-a99c859518bb admin admin] RBAC: Authorizing `identity:create_grant(project_id=8586f82366634d49a9155468fd73e451, user_id=875072bd53cc4b4d9842f8a89a47fdc8, role_id=6876e57cc51c448097ab109d35b2a3da)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:31:44.259748 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee574e2-fb06-427f-b8ff-a99c859518bb admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:31:44.272349 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329126f3-ce2d-450c-a108-f9c0229abc8b admin admin] RBAC: Authorizing `identity:create_grant(project_id=c4d17fc4571545cebec33d03b677a6c1, user_id=4798b64da9a54711a2edf6679fd4f32e, role_id=6876e57cc51c448097ab109d35b2a3da)` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:31:44.276236 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329126f3-ce2d-450c-a108-f9c0229abc8b admin admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:31:44.283952 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 898/1795] 10.222.0.49 () {60 vars in 1482 bytes} [Wed Jun 15 10:31:44 2022] PUT /identity/v3/projects/8586f82366634d49a9155468fd73e451/users/875072bd53cc4b4d9842f8a89a47fdc8/roles/6876e57cc51c448097ab109d35b2a3da => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:31:44.307081 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a93c1a-cb14-4ede-ac71-63ead10d7df7 None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:31:44.307353 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a93c1a-cb14-4ede-ac71-63ead10d7df7 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:31:44.307666 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a93c1a-cb14-4ede-ac71-63ead10d7df7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:31:44.325945 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 898/1796] 10.222.0.49 () {60 vars in 1482 bytes} [Wed Jun 15 10:31:44 2022] PUT /identity/v3/projects/c4d17fc4571545cebec33d03b677a6c1/users/4798b64da9a54711a2edf6679fd4f32e/roles/6876e57cc51c448097ab109d35b2a3da => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:31:44.338023 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff45ca1-c26f-4a9f-9fc4-dbe0c3ae45dd None None] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:31:44.340504 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff45ca1-c26f-4a9f-9fc4-dbe0c3ae45dd None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:31:44.341293 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff45ca1-c26f-4a9f-9fc4-dbe0c3ae45dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:31:44.345787 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-69a93c1a-cb14-4ede-ac71-63ead10d7df7 None None] MFA Rules not processed for user `14b720a2d0854d218272390205887100`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:31:44.351748 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-69a93c1a-cb14-4ede-ac71-63ead10d7df7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:31:44.434429 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.auth.core [None req-dff45ca1-c26f-4a9f-9fc4-dbe0c3ae45dd None None] MFA Rules not processed for user `875072bd53cc4b4d9842f8a89a47fdc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:31:44.436156 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-dff45ca1-c26f-4a9f-9fc4-dbe0c3ae45dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:31:44.496873 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 899/1797] 10.222.0.49 () {58 vars in 957 bytes} [Wed Jun 15 10:31:44 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:31:44.498321 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec175ac-120f-4fff-823c-9a897418c7cb None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:31:44.498622 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec175ac-120f-4fff-823c-9a897418c7cb None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:31:44.498907 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec175ac-120f-4fff-823c-9a897418c7cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:31:44.556125 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-4ec175ac-120f-4fff-823c-9a897418c7cb None None] MFA Rules not processed for user `4798b64da9a54711a2edf6679fd4f32e`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:31:44.557380 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-4ec175ac-120f-4fff-823c-9a897418c7cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:31:44.583000 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 899/1798] 10.222.0.49 () {58 vars in 957 bytes} [Wed Jun 15 10:31:44 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 256 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:31:44.587344 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff45ca1-c26f-4a9f-9fc4-dbe0c3ae45dd None None] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:31:44.656201 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:31:44.712338 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff45ca1-c26f-4a9f-9fc4-dbe0c3ae45dd None None] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:31:44.720224 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b22057-4ffc-4915-9a24-ddad2c6786eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:31:44.721233 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b22057-4ffc-4915-9a24-ddad2c6786eb admin admin] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:31:44.721950 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b22057-4ffc-4915-9a24-ddad2c6786eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:31:44.722429 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b22057-4ffc-4915-9a24-ddad2c6786eb admin admin] PATH_INFO: `/v3/projects` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:31:44.724178 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b22057-4ffc-4915-9a24-ddad2c6786eb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:31:44.726381 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b22057-4ffc-4915-9a24-ddad2c6786eb admin admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:31:44.781566 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 900/1799] 10.222.0.49 () {60 vars in 1152 bytes} [Wed Jun 15 10:31:44 2022] POST /identity/v3/projects => generated 405 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 10:31:44.794042 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b22057-4ffc-4915-9a24-ddad2c6786eb admin admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:31:44.796856 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 900/1800] 10.222.0.49 () {58 vars in 957 bytes} [Wed Jun 15 10:31:44 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 299 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:31:44.801238 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec175ac-120f-4fff-823c-9a897418c7cb None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:31:44.817777 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec175ac-120f-4fff-823c-9a897418c7cb None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:31:44.858880 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:31:44.898651 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b22057-4ffc-4915-9a24-ddad2c6786eb admin admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:31:44.907380 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667a998b-218a-4f09-80b5-8533fe14f589 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:31:44.908435 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667a998b-218a-4f09-80b5-8533fe14f589 admin admin] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:31:44.908832 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667a998b-218a-4f09-80b5-8533fe14f589 admin admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:31:44.909173 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667a998b-218a-4f09-80b5-8533fe14f589 admin admin] PATH_INFO: `/v3/users` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:31:44.909949 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-667a998b-218a-4f09-80b5-8533fe14f589 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:31:44.912135 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-667a998b-218a-4f09-80b5-8533fe14f589 admin admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:31:44.943704 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 901/1801] 10.222.0.49 () {60 vars in 1143 bytes} [Wed Jun 15 10:31:44 2022] POST /identity/v3/users => generated 309 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 10:31:44.948391 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667a998b-218a-4f09-80b5-8533fe14f589 admin admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:31:44.956026 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f30410-9e4f-4d66-8554-1b2eae8248ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:31:44.958041 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f30410-9e4f-4d66-8554-1b2eae8248ac admin admin] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:31:44.958886 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f30410-9e4f-4d66-8554-1b2eae8248ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:31:44.959454 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f30410-9e4f-4d66-8554-1b2eae8248ac admin admin] PATH_INFO: `/v3/projects` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:31:44.960634 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f30410-9e4f-4d66-8554-1b2eae8248ac admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:31:44.962522 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667a998b-218a-4f09-80b5-8533fe14f589 admin admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:31:44.963346 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f30410-9e4f-4d66-8554-1b2eae8248ac admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:31:44.969699 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00fb526d-eedf-4c0f-a82f-8eb7bdddff7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:31:44.970600 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fb526d-eedf-4c0f-a82f-8eb7bdddff7c admin admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:31:44.972460 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fb526d-eedf-4c0f-a82f-8eb7bdddff7c admin admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:31:44.972914 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fb526d-eedf-4c0f-a82f-8eb7bdddff7c admin admin] PATH_INFO: `/v3/roles` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:31:44.973881 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00fb526d-eedf-4c0f-a82f-8eb7bdddff7c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:31:44.976355 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00fb526d-eedf-4c0f-a82f-8eb7bdddff7c admin admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:31:44.994713 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 902/1802] 10.222.0.49 () {58 vars in 1121 bytes} [Wed Jun 15 10:31:44 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 10:31:44.999510 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00fb526d-eedf-4c0f-a82f-8eb7bdddff7c admin admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:31:45.011309 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 901/1803] 10.222.0.49 () {60 vars in 1152 bytes} [Wed Jun 15 10:31:44 2022] POST /identity/v3/projects => generated 367 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 10:31:45.015588 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f30410-9e4f-4d66-8554-1b2eae8248ac admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:31:45.025057 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00fb526d-eedf-4c0f-a82f-8eb7bdddff7c admin admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:31:45.126946 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287b9c17-097b-4845-a719-f68c8f24baf4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:31:45.127884 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287b9c17-097b-4845-a719-f68c8f24baf4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:31:45.128168 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287b9c17-097b-4845-a719-f68c8f24baf4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:31:45.128445 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287b9c17-097b-4845-a719-f68c8f24baf4 admin admin] PATH_INFO: `/v3/projects/4150b5d8f39d4f8f81738e8a2c427d48/users/42a3e7682fa34727ac3dc9b6da45ae89/roles/8051174e7a7947d5b991ebd793c2e543` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:31:45.162536 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f30410-9e4f-4d66-8554-1b2eae8248ac admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:31:45.166771 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-287b9c17-097b-4845-a719-f68c8f24baf4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4150b5d8f39d4f8f81738e8a2c427d48, user_id=42a3e7682fa34727ac3dc9b6da45ae89, role_id=8051174e7a7947d5b991ebd793c2e543)` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:31:45.169586 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-287b9c17-097b-4845-a719-f68c8f24baf4 admin admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:31:45.177230 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d324aa13-9153-4270-9ecb-aa7e5eca39d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:31:45.177933 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d324aa13-9153-4270-9ecb-aa7e5eca39d1 admin admin] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:31:45.178221 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d324aa13-9153-4270-9ecb-aa7e5eca39d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:31:45.178501 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d324aa13-9153-4270-9ecb-aa7e5eca39d1 admin admin] PATH_INFO: `/v3/users` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:31:45.179298 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d324aa13-9153-4270-9ecb-aa7e5eca39d1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:31:45.182416 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d324aa13-9153-4270-9ecb-aa7e5eca39d1 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:31:45.195895 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 903/1804] 10.222.0.49 () {60 vars in 1482 bytes} [Wed Jun 15 10:31:44 2022] PUT /identity/v3/projects/4150b5d8f39d4f8f81738e8a2c427d48/users/42a3e7682fa34727ac3dc9b6da45ae89/roles/8051174e7a7947d5b991ebd793c2e543 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:31:45.199493 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287b9c17-097b-4845-a719-f68c8f24baf4 admin admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:31:45.227717 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 902/1805] 10.222.0.49 () {60 vars in 1143 bytes} [Wed Jun 15 10:31:45 2022] POST /identity/v3/users => generated 290 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 10:31:45.228969 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d324aa13-9153-4270-9ecb-aa7e5eca39d1 admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:31:45.252114 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d324aa13-9153-4270-9ecb-aa7e5eca39d1 admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:31:45.348601 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:31:45.379615 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bfa71bb-7f77-40a9-bc26-f7eace37bae1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:31:45.380546 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfa71bb-7f77-40a9-bc26-f7eace37bae1 admin admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:31:45.381034 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfa71bb-7f77-40a9-bc26-f7eace37bae1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:31:45.381391 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfa71bb-7f77-40a9-bc26-f7eace37bae1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:31:45.382250 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bfa71bb-7f77-40a9-bc26-f7eace37bae1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:31:45.384319 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bfa71bb-7f77-40a9-bc26-f7eace37bae1 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:31:45.391253 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287b9c17-097b-4845-a719-f68c8f24baf4 admin admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:31:45.404435 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2cfdc4-216e-42b7-bb72-7ff7eb65e23b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:31:45.405852 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2cfdc4-216e-42b7-bb72-7ff7eb65e23b admin admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:31:45.407345 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 903/1806] 10.222.0.49 () {58 vars in 1121 bytes} [Wed Jun 15 10:31:45 2022] GET /identity/v3/roles => generated 1420 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 15 10:31:45.408668 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bfa71bb-7f77-40a9-bc26-f7eace37bae1 admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:31:45.409221 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2cfdc4-216e-42b7-bb72-7ff7eb65e23b admin admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:31:45.409915 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2cfdc4-216e-42b7-bb72-7ff7eb65e23b admin admin] PATH_INFO: `/v3/roles` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:31:45.412411 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d2cfdc4-216e-42b7-bb72-7ff7eb65e23b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:31:45.418325 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d2cfdc4-216e-42b7-bb72-7ff7eb65e23b admin admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:31:45.439244 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bfa71bb-7f77-40a9-bc26-f7eace37bae1 admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:31:45.456233 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ebd050-f283-462e-9022-e67401a49d4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:31:45.456233 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ebd050-f283-462e-9022-e67401a49d4a admin admin] REQUEST_METHOD: `PUT` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:31:45.456233 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ebd050-f283-462e-9022-e67401a49d4a admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:31:45.456233 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ebd050-f283-462e-9022-e67401a49d4a admin admin] PATH_INFO: `/v3/projects/5b00baf3d6d940bfa8531957dcb0b7e2/users/281bbce0f25a40cf959f30d25c61945f/roles/8051174e7a7947d5b991ebd793c2e543` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:31:45.460691 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 904/1807] 10.222.0.49 () {58 vars in 1121 bytes} [Wed Jun 15 10:31:45 2022] GET /identity/v3/roles => generated 1420 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 10:31:45.462765 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2cfdc4-216e-42b7-bb72-7ff7eb65e23b admin admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:31:45.487374 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2cfdc4-216e-42b7-bb72-7ff7eb65e23b admin admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:31:45.502827 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e70f70-2a0c-4afb-9ab4-5c27c3ed20c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:31:45.504338 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e70f70-2a0c-4afb-9ab4-5c27c3ed20c0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:31:45.504868 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e70f70-2a0c-4afb-9ab4-5c27c3ed20c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:31:45.505247 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ebd050-f283-462e-9022-e67401a49d4a admin admin] RBAC: Authorizing `identity:create_grant(project_id=5b00baf3d6d940bfa8531957dcb0b7e2, user_id=281bbce0f25a40cf959f30d25c61945f, role_id=8051174e7a7947d5b991ebd793c2e543)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:31:45.505504 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e70f70-2a0c-4afb-9ab4-5c27c3ed20c0 admin admin] PATH_INFO: `/v3/projects/4150b5d8f39d4f8f81738e8a2c427d48/users/42a3e7682fa34727ac3dc9b6da45ae89/roles/8051174e7a7947d5b991ebd793c2e543` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:31:45.508151 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ebd050-f283-462e-9022-e67401a49d4a admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:31:45.510919 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e70f70-2a0c-4afb-9ab4-5c27c3ed20c0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4150b5d8f39d4f8f81738e8a2c427d48, user_id=42a3e7682fa34727ac3dc9b6da45ae89, role_id=8051174e7a7947d5b991ebd793c2e543)` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:31:45.514165 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e70f70-2a0c-4afb-9ab4-5c27c3ed20c0 admin admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:31:45.538616 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 904/1809] 10.222.0.49 () {60 vars in 1482 bytes} [Wed Jun 15 10:31:45 2022] PUT /identity/v3/projects/5b00baf3d6d940bfa8531957dcb0b7e2/users/281bbce0f25a40cf959f30d25c61945f/roles/8051174e7a7947d5b991ebd793c2e543 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:31:45.540939 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ebd050-f283-462e-9022-e67401a49d4a admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:31:45.544583 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 905/1809] 10.222.0.49 () {60 vars in 1482 bytes} [Wed Jun 15 10:31:45 2022] PUT /identity/v3/projects/4150b5d8f39d4f8f81738e8a2c427d48/users/42a3e7682fa34727ac3dc9b6da45ae89/roles/8051174e7a7947d5b991ebd793c2e543 => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:31:45.548047 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7c32bd-5383-47f8-86aa-63c49704618f None None] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:31:45.548601 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7c32bd-5383-47f8-86aa-63c49704618f None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:31:45.548949 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7c32bd-5383-47f8-86aa-63c49704618f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:31:45.618158 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.auth.core [None req-7e7c32bd-5383-47f8-86aa-63c49704618f None None] MFA Rules not processed for user `42a3e7682fa34727ac3dc9b6da45ae89`. Rule list: `[]` (Enabled: `True`). {{(pid=80119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:31:45.619880 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-7e7c32bd-5383-47f8-86aa-63c49704618f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:31:45.651714 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:31:45.697527 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ebd050-f283-462e-9022-e67401a49d4a admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:31:45.724487 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c214b24b-6009-4ea8-b488-deead6082ad7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:31:45.725522 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c214b24b-6009-4ea8-b488-deead6082ad7 admin admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:31:45.726141 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c214b24b-6009-4ea8-b488-deead6082ad7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:31:45.726584 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c214b24b-6009-4ea8-b488-deead6082ad7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:31:45.728894 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c214b24b-6009-4ea8-b488-deead6082ad7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:31:45.733694 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c214b24b-6009-4ea8-b488-deead6082ad7 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:31:45.838744 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 905/1810] 10.222.0.49 () {58 vars in 1121 bytes} [Wed Jun 15 10:31:45 2022] GET /identity/v3/roles => generated 1420 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 10:31:45.848137 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd27d44-7083-46ef-9348-8af19015ef24 None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:31:45.848430 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd27d44-7083-46ef-9348-8af19015ef24 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:31:45.848711 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd27d44-7083-46ef-9348-8af19015ef24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:31:45.856623 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 906/1811] 10.222.0.49 () {58 vars in 957 bytes} [Wed Jun 15 10:31:45 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 311 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:31:45.873160 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7c32bd-5383-47f8-86aa-63c49704618f None None] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:31:45.887667 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7c32bd-5383-47f8-86aa-63c49704618f None None] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:31:45.895452 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a94e86-3a4c-4ee7-9096-038e5753abc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:31:45.896401 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a94e86-3a4c-4ee7-9096-038e5753abc6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:31:45.896867 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a94e86-3a4c-4ee7-9096-038e5753abc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:31:45.897149 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a94e86-3a4c-4ee7-9096-038e5753abc6 admin admin] PATH_INFO: `/v3/projects/5b00baf3d6d940bfa8531957dcb0b7e2/users/281bbce0f25a40cf959f30d25c61945f/roles/8051174e7a7947d5b991ebd793c2e543` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:31:45.900572 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a94e86-3a4c-4ee7-9096-038e5753abc6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5b00baf3d6d940bfa8531957dcb0b7e2, user_id=281bbce0f25a40cf959f30d25c61945f, role_id=8051174e7a7947d5b991ebd793c2e543)` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:31:45.903895 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a94e86-3a4c-4ee7-9096-038e5753abc6 admin admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:31:45.917658 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 907/1812] 10.222.0.49 () {60 vars in 1482 bytes} [Wed Jun 15 10:31:45 2022] PUT /identity/v3/projects/5b00baf3d6d940bfa8531957dcb0b7e2/users/281bbce0f25a40cf959f30d25c61945f/roles/8051174e7a7947d5b991ebd793c2e543 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:31:45.918996 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4324b4b6-9d6a-4251-95f4-995ddc213b4a None None] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:31:45.919294 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4324b4b6-9d6a-4251-95f4-995ddc213b4a None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:31:45.919576 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4324b4b6-9d6a-4251-95f4-995ddc213b4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:31:45.934441 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-6cd27d44-7083-46ef-9348-8af19015ef24 None None] MFA Rules not processed for user `318f2b2a3f024c7a8e56269feb343d9f`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:31:45.935709 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-6cd27d44-7083-46ef-9348-8af19015ef24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:31:45.965161 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.auth.core [None req-4324b4b6-9d6a-4251-95f4-995ddc213b4a None None] MFA Rules not processed for user `cca7cb5e7620446ebae91b1fe12bbd24`. Rule list: `[]` (Enabled: `True`). {{(pid=80119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:31:45.967774 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-4324b4b6-9d6a-4251-95f4-995ddc213b4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:31:46.060267 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 908/1813] 10.222.0.49 () {58 vars in 957 bytes} [Wed Jun 15 10:31:45 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:31:46.061462 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4324b4b6-9d6a-4251-95f4-995ddc213b4a None None] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:31:46.062962 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-4324b4b6-9d6a-4251-95f4-995ddc213b4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:31:46.064671 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 906/1814] 10.222.0.49 () {58 vars in 957 bytes} [Wed Jun 15 10:31:45 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:31:46.066343 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51a0603-86cb-4a01-90cf-0a7bfb1166e8 None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:31:46.066646 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51a0603-86cb-4a01-90cf-0a7bfb1166e8 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:31:46.066935 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51a0603-86cb-4a01-90cf-0a7bfb1166e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:31:46.108783 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-c51a0603-86cb-4a01-90cf-0a7bfb1166e8 None None] MFA Rules not processed for user `281bbce0f25a40cf959f30d25c61945f`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:31:46.110198 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-c51a0603-86cb-4a01-90cf-0a7bfb1166e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:31:46.144839 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4324b4b6-9d6a-4251-95f4-995ddc213b4a None None] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:31:46.148602 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f1dd51-fa7b-4263-b909-5bae2d503589 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20d1327553404b80aecee87c78fcb52b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '559b3e713f5844b6ae37151d56dd419f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:31:46.150011 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f1dd51-fa7b-4263-b909-5bae2d503589 service cinder] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:31:46.150376 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f1dd51-fa7b-4263-b909-5bae2d503589 service cinder] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:31:46.150781 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f1dd51-fa7b-4263-b909-5bae2d503589 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:31:46.163971 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:31:46.211952 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 907/1815] 10.222.0.49 () {58 vars in 957 bytes} [Wed Jun 15 10:31:46 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:31:46.213509 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51a0603-86cb-4a01-90cf-0a7bfb1166e8 None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:31:46.215353 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-c51a0603-86cb-4a01-90cf-0a7bfb1166e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:31:46.223002 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:31:46.239197 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f1dd51-fa7b-4263-b909-5bae2d503589 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:31:46.242053 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f1dd51-fa7b-4263-b909-5bae2d503589 service cinder] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:31:46.254591 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 909/1816] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Jun 15 10:31:46 2022] GET /identity/v3/auth/tokens => generated 3339 bytes in 193 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:31:46.255472 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f1dd51-fa7b-4263-b909-5bae2d503589 service cinder] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:31:46.265347 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f1dd51-fa7b-4263-b909-5bae2d503589 service cinder] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:31:46.269655 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1aa4783-d0db-49d8-9497-b7816782ab07 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20d1327553404b80aecee87c78fcb52b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '559b3e713f5844b6ae37151d56dd419f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:31:46.270376 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1aa4783-d0db-49d8-9497-b7816782ab07 service cinder] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:31:46.270670 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1aa4783-d0db-49d8-9497-b7816782ab07 service cinder] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:31:46.271618 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1aa4783-d0db-49d8-9497-b7816782ab07 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:31:46.283885 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1aa4783-d0db-49d8-9497-b7816782ab07 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:31:46.285410 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1aa4783-d0db-49d8-9497-b7816782ab07 service cinder] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:31:46.297832 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 910/1817] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Jun 15 10:31:46 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:31:46.365925 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51a0603-86cb-4a01-90cf-0a7bfb1166e8 None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:31:46.388890 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29fdf63-dad6-42bc-85a2-78edb5478f9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:31:46.389999 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29fdf63-dad6-42bc-85a2-78edb5478f9f admin admin] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:31:46.390537 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29fdf63-dad6-42bc-85a2-78edb5478f9f admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:31:46.390890 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29fdf63-dad6-42bc-85a2-78edb5478f9f admin admin] PATH_INFO: `/v3/projects` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:31:46.391738 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29fdf63-dad6-42bc-85a2-78edb5478f9f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:31:46.394619 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29fdf63-dad6-42bc-85a2-78edb5478f9f admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:31:46.463934 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 908/1818] 10.222.0.49 () {60 vars in 1152 bytes} [Wed Jun 15 10:31:46 2022] POST /identity/v3/projects => generated 405 bytes in 249 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 10:31:46.468034 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1aa4783-d0db-49d8-9497-b7816782ab07 service cinder] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:31:46.475998 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29fdf63-dad6-42bc-85a2-78edb5478f9f admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:31:46.482733 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1aa4783-d0db-49d8-9497-b7816782ab07 service cinder] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:31:46.560980 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3470d2c0-1dae-4026-bfea-def8065d7a47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:31:46.561685 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3470d2c0-1dae-4026-bfea-def8065d7a47 admin admin] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:31:46.561968 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3470d2c0-1dae-4026-bfea-def8065d7a47 admin admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:31:46.562246 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3470d2c0-1dae-4026-bfea-def8065d7a47 admin admin] PATH_INFO: `/v3/users` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:31:46.562940 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3470d2c0-1dae-4026-bfea-def8065d7a47 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:31:46.564968 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3470d2c0-1dae-4026-bfea-def8065d7a47 admin admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:31:46.606422 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 911/1819] 10.222.0.49 () {60 vars in 1143 bytes} [Wed Jun 15 10:31:46 2022] POST /identity/v3/users => generated 309 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 10:31:46.610840 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3470d2c0-1dae-4026-bfea-def8065d7a47 admin admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:31:46.623137 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3470d2c0-1dae-4026-bfea-def8065d7a47 admin admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:31:46.629563 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6dede4-6d20-42fc-b283-9031f3526f44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:31:46.630510 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6dede4-6d20-42fc-b283-9031f3526f44 admin admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:31:46.630795 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6dede4-6d20-42fc-b283-9031f3526f44 admin admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:31:46.631106 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6dede4-6d20-42fc-b283-9031f3526f44 admin admin] PATH_INFO: `/v3/roles` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:31:46.631914 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6dede4-6d20-42fc-b283-9031f3526f44 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:31:46.634271 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6dede4-6d20-42fc-b283-9031f3526f44 admin admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:31:46.648119 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 912/1820] 10.222.0.49 () {58 vars in 1121 bytes} [Wed Jun 15 10:31:46 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 10:31:46.653424 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6dede4-6d20-42fc-b283-9031f3526f44 admin admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:31:46.665355 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6dede4-6d20-42fc-b283-9031f3526f44 admin admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:31:46.670292 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e406fd-026c-4618-855d-9148115beb23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:31:46.671391 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e406fd-026c-4618-855d-9148115beb23 admin admin] REQUEST_METHOD: `PUT` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:31:46.671673 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e406fd-026c-4618-855d-9148115beb23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:31:46.671953 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e406fd-026c-4618-855d-9148115beb23 admin admin] PATH_INFO: `/v3/projects/925619655cc54cc29e408533cec49d9e/users/2fc5998233d146a3ba9701410af7bc58/roles/8051174e7a7947d5b991ebd793c2e543` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:31:46.693059 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e406fd-026c-4618-855d-9148115beb23 admin admin] RBAC: Authorizing `identity:create_grant(project_id=925619655cc54cc29e408533cec49d9e, user_id=2fc5998233d146a3ba9701410af7bc58, role_id=8051174e7a7947d5b991ebd793c2e543)` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:31:46.696562 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e406fd-026c-4618-855d-9148115beb23 admin admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:31:46.702013 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29fdf63-dad6-42bc-85a2-78edb5478f9f admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:31:46.706784 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48ef5fe-5bdb-4c71-9f7a-96b6c74a5db2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20d1327553404b80aecee87c78fcb52b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '559b3e713f5844b6ae37151d56dd419f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:31:46.707501 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48ef5fe-5bdb-4c71-9f7a-96b6c74a5db2 service cinder] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:31:46.707927 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48ef5fe-5bdb-4c71-9f7a-96b6c74a5db2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:31:46.708214 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48ef5fe-5bdb-4c71-9f7a-96b6c74a5db2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:31:46.720642 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 913/1821] 10.222.0.49 () {60 vars in 1482 bytes} [Wed Jun 15 10:31:46 2022] PUT /identity/v3/projects/925619655cc54cc29e408533cec49d9e/users/2fc5998233d146a3ba9701410af7bc58/roles/8051174e7a7947d5b991ebd793c2e543 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 10:31:46.725266 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e406fd-026c-4618-855d-9148115beb23 admin admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:31:46.782084 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b48ef5fe-5bdb-4c71-9f7a-96b6c74a5db2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:31:46.783850 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b48ef5fe-5bdb-4c71-9f7a-96b6c74a5db2 service cinder] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:31:46.800602 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 909/1822] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Jun 15 10:31:46 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 10:31:46.829864 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e406fd-026c-4618-855d-9148115beb23 admin admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:31:46.835793 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cebe6854-d951-4c51-938c-ebb3a94703fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:31:46.836693 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebe6854-d951-4c51-938c-ebb3a94703fd admin admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:31:46.838299 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebe6854-d951-4c51-938c-ebb3a94703fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:31:46.838661 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebe6854-d951-4c51-938c-ebb3a94703fd admin admin] PATH_INFO: `/v3/roles` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:31:46.840283 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cebe6854-d951-4c51-938c-ebb3a94703fd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:31:46.845743 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cebe6854-d951-4c51-938c-ebb3a94703fd admin admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:31:46.865059 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 914/1823] 10.222.0.49 () {58 vars in 1121 bytes} [Wed Jun 15 10:31:46 2022] GET /identity/v3/roles => generated 1420 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 10:31:46.866367 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48ef5fe-5bdb-4c71-9f7a-96b6c74a5db2 service cinder] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:31:46.880242 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48ef5fe-5bdb-4c71-9f7a-96b6c74a5db2 service cinder] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:31:46.886236 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c7b65a-37cb-46ff-be97-4b026045565c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:31:46.887684 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c7b65a-37cb-46ff-be97-4b026045565c admin admin] REQUEST_METHOD: `PUT` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:31:46.887815 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c7b65a-37cb-46ff-be97-4b026045565c admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:31:46.888135 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c7b65a-37cb-46ff-be97-4b026045565c admin admin] PATH_INFO: `/v3/projects/925619655cc54cc29e408533cec49d9e/users/2fc5998233d146a3ba9701410af7bc58/roles/8051174e7a7947d5b991ebd793c2e543` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:31:46.890549 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c7b65a-37cb-46ff-be97-4b026045565c admin admin] RBAC: Authorizing `identity:create_grant(project_id=925619655cc54cc29e408533cec49d9e, user_id=2fc5998233d146a3ba9701410af7bc58, role_id=8051174e7a7947d5b991ebd793c2e543)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:31:46.893756 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c7b65a-37cb-46ff-be97-4b026045565c admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:31:46.902165 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:31:46.911170 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 910/1824] 10.222.0.49 () {60 vars in 1482 bytes} [Wed Jun 15 10:31:46 2022] PUT /identity/v3/projects/925619655cc54cc29e408533cec49d9e/users/2fc5998233d146a3ba9701410af7bc58/roles/8051174e7a7947d5b991ebd793c2e543 => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:31:47.947141 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a161272b-93bc-42d7-aea5-456696fb997d None None] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:31:47.947706 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a161272b-93bc-42d7-aea5-456696fb997d None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:31:47.947905 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a161272b-93bc-42d7-aea5-456696fb997d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:31:47.975483 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.auth.core [None req-a161272b-93bc-42d7-aea5-456696fb997d None None] MFA Rules not processed for user `2fc5998233d146a3ba9701410af7bc58`. Rule list: `[]` (Enabled: `True`). {{(pid=80119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:31:47.977528 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-a161272b-93bc-42d7-aea5-456696fb997d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:31:47.995684 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:31:48.175972 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 915/1825] 10.222.0.49 () {58 vars in 957 bytes} [Wed Jun 15 10:31:47 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:31:48.785386 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c7b65a-37cb-46ff-be97-4b026045565c admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:31:48.856572 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c7b65a-37cb-46ff-be97-4b026045565c admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:31:48.861371 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aee3fa8-5d3f-4467-a84c-0c134561fa24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:31:48.862300 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aee3fa8-5d3f-4467-a84c-0c134561fa24 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:31:48.862591 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aee3fa8-5d3f-4467-a84c-0c134561fa24 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:31:48.862870 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aee3fa8-5d3f-4467-a84c-0c134561fa24 admin admin] PATH_INFO: `/v3/users/875072bd53cc4b4d9842f8a89a47fdc8` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:31:48.864761 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aee3fa8-5d3f-4467-a84c-0c134561fa24 admin admin] RBAC: Authorizing `identity:delete_user(user_id=875072bd53cc4b4d9842f8a89a47fdc8)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:31:48.866819 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aee3fa8-5d3f-4467-a84c-0c134561fa24 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:31:48.915444 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-8aee3fa8-5d3f-4467-a84c-0c134561fa24 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '875072bd53cc4b4d9842f8a89a47fdc8'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:31:48.922705 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-8aee3fa8-5d3f-4467-a84c-0c134561fa24 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '875072bd53cc4b4d9842f8a89a47fdc8'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:31:48.923409 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-8aee3fa8-5d3f-4467-a84c-0c134561fa24 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '875072bd53cc4b4d9842f8a89a47fdc8'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:31:48.924041 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-8aee3fa8-5d3f-4467-a84c-0c134561fa24 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '875072bd53cc4b4d9842f8a89a47fdc8'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:31:48.933189 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-8aee3fa8-5d3f-4467-a84c-0c134561fa24 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '875072bd53cc4b4d9842f8a89a47fdc8'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:31:48.951147 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 911/1826] 10.222.0.49 () {58 vars in 1223 bytes} [Wed Jun 15 10:31:48 2022] DELETE /identity/v3/users/875072bd53cc4b4d9842f8a89a47fdc8 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:31:48.955843 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea57d38-4886-444b-9734-865f3ddb0b1b None None] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:31:48.956349 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea57d38-4886-444b-9734-865f3ddb0b1b None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:31:48.956440 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea57d38-4886-444b-9734-865f3ddb0b1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:31:48.974698 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.auth.core [None req-7ea57d38-4886-444b-9734-865f3ddb0b1b None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:31:48.976137 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-7ea57d38-4886-444b-9734-865f3ddb0b1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:31:49.037162 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 916/1827] 10.222.0.49 () {58 vars in 957 bytes} [Wed Jun 15 10:31:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:31:49.048006 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aee3fa8-5d3f-4467-a84c-0c134561fa24 admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:31:49.049506 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-8aee3fa8-5d3f-4467-a84c-0c134561fa24 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:31:49.097155 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6539db44-2a18-4cba-99ee-02ee76ee6c21 None None] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:31:49.097876 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6539db44-2a18-4cba-99ee-02ee76ee6c21 None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:31:49.098260 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6539db44-2a18-4cba-99ee-02ee76ee6c21 None None] PATH_INFO: `/` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:31:49.101302 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 917/1828] 10.222.0.49 () {54 vars in 921 bytes} [Wed Jun 15 10:31:49 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 15 10:31:49.115670 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aee3fa8-5d3f-4467-a84c-0c134561fa24 admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:31:49.119382 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549b267c-c12d-491d-b3d1-8086023d4155 None None] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:31:49.119690 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549b267c-c12d-491d-b3d1-8086023d4155 None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:31:49.119957 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549b267c-c12d-491d-b3d1-8086023d4155 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:31:49.120837 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a8805a-1b8c-4f36-8b7b-007b8b51ba44 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0cb44a17d205431793e9a8791e477a32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '559b3e713f5844b6ae37151d56dd419f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:31:49.121658 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a8805a-1b8c-4f36-8b7b-007b8b51ba44 service neutron] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:31:49.122129 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a8805a-1b8c-4f36-8b7b-007b8b51ba44 service neutron] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:31:49.122564 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a8805a-1b8c-4f36-8b7b-007b8b51ba44 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:31:49.145619 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.auth.core [None req-549b267c-c12d-491d-b3d1-8086023d4155 None None] MFA Rules not processed for user `dd03612a525644518517069b3fa43522`. Rule list: `[]` (Enabled: `True`). {{(pid=80119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:31:49.146031 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a8805a-1b8c-4f36-8b7b-007b8b51ba44 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:31:49.147353 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-549b267c-c12d-491d-b3d1-8086023d4155 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:31:49.147817 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a8805a-1b8c-4f36-8b7b-007b8b51ba44 service neutron] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:31:49.162168 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 912/1829] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Jun 15 10:31:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:31:49.238886 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 918/1830] 10.222.0.49 () {58 vars in 1018 bytes} [Wed Jun 15 10:31:49 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:31:49.251687 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a8805a-1b8c-4f36-8b7b-007b8b51ba44 service neutron] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:31:49.253497 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-39a8805a-1b8c-4f36-8b7b-007b8b51ba44 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:31:49.272437 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a8805a-1b8c-4f36-8b7b-007b8b51ba44 service neutron] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:31:49.279557 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5172e38-9403-4b1f-92bc-55b1496d1cc3 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd03612a525644518517069b3fa43522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '559b3e713f5844b6ae37151d56dd419f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:31:49.280599 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5172e38-9403-4b1f-92bc-55b1496d1cc3 service placement] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:31:49.280898 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5172e38-9403-4b1f-92bc-55b1496d1cc3 service placement] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:31:49.281178 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5172e38-9403-4b1f-92bc-55b1496d1cc3 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:31:49.294247 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5172e38-9403-4b1f-92bc-55b1496d1cc3 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:31:49.296542 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5172e38-9403-4b1f-92bc-55b1496d1cc3 service placement] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:31:49.310613 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 913/1831] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Jun 15 10:31:49 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:31:49.757964 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549b267c-c12d-491d-b3d1-8086023d4155 None None] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:31:49.759730 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-549b267c-c12d-491d-b3d1-8086023d4155 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:31:49.792247 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549b267c-c12d-491d-b3d1-8086023d4155 None None] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:31:49.801793 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33869886-dcff-4eaa-96b8-afa091d46a41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:31:49.807519 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33869886-dcff-4eaa-96b8-afa091d46a41 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:31:49.808615 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33869886-dcff-4eaa-96b8-afa091d46a41 admin admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:31:49.809024 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33869886-dcff-4eaa-96b8-afa091d46a41 admin admin] PATH_INFO: `/v3/projects/8586f82366634d49a9155468fd73e451` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:31:49.813864 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33869886-dcff-4eaa-96b8-afa091d46a41 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8586f82366634d49a9155468fd73e451)` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:31:49.817528 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33869886-dcff-4eaa-96b8-afa091d46a41 admin admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:31:49.892680 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:31:49.896900 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-33869886-dcff-4eaa-96b8-afa091d46a41 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8586f82366634d49a9155468fd73e451', 'request_id': 'req-33869886-dcff-4eaa-96b8-afa091d46a41'} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:31:49.912808 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-33869886-dcff-4eaa-96b8-afa091d46a41 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8586f82366634d49a9155468fd73e451', 'request_id': 'req-33869886-dcff-4eaa-96b8-afa091d46a41'} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:31:49.921024 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-33869886-dcff-4eaa-96b8-afa091d46a41 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8586f82366634d49a9155468fd73e451', 'request_id': 'req-33869886-dcff-4eaa-96b8-afa091d46a41'} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:31:49.938941 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-33869886-dcff-4eaa-96b8-afa091d46a41 admin admin] The token cache is being invalidate because project 8586f82366634d49a9155468fd73e451 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80119) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 10:31:49.939305 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-33869886-dcff-4eaa-96b8-afa091d46a41 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:31:49.940017 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-33869886-dcff-4eaa-96b8-afa091d46a41 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:31:49.941850 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 919/1832] 10.222.0.49 () {58 vars in 1232 bytes} [Wed Jun 15 10:31:49 2022] DELETE /identity/v3/projects/8586f82366634d49a9155468fd73e451 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:31:49.946443 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5172e38-9403-4b1f-92bc-55b1496d1cc3 service placement] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:31:49.958800 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5172e38-9403-4b1f-92bc-55b1496d1cc3 service placement] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:31:49.963832 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:31:50.019792 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24a0bf4-40a4-4241-ba8d-34316d346871 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:31:50.020683 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24a0bf4-40a4-4241-ba8d-34316d346871 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:31:50.020967 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24a0bf4-40a4-4241-ba8d-34316d346871 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:31:50.021243 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24a0bf4-40a4-4241-ba8d-34316d346871 admin admin] PATH_INFO: `/v3/users/42a3e7682fa34727ac3dc9b6da45ae89` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:31:50.022440 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24a0bf4-40a4-4241-ba8d-34316d346871 admin admin] RBAC: Authorizing `identity:delete_user(user_id=42a3e7682fa34727ac3dc9b6da45ae89)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:31:50.024626 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24a0bf4-40a4-4241-ba8d-34316d346871 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:31:50.077808 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-d24a0bf4-40a4-4241-ba8d-34316d346871 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '42a3e7682fa34727ac3dc9b6da45ae89'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:31:50.085835 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-d24a0bf4-40a4-4241-ba8d-34316d346871 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '42a3e7682fa34727ac3dc9b6da45ae89'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:31:50.086686 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-d24a0bf4-40a4-4241-ba8d-34316d346871 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '42a3e7682fa34727ac3dc9b6da45ae89'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:31:50.087409 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-d24a0bf4-40a4-4241-ba8d-34316d346871 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '42a3e7682fa34727ac3dc9b6da45ae89'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:31:50.098002 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-d24a0bf4-40a4-4241-ba8d-34316d346871 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '42a3e7682fa34727ac3dc9b6da45ae89'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:31:50.103437 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69037dab-7a4a-4b43-8fdb-fa02c56d2262 None None] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:31:50.104457 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69037dab-7a4a-4b43-8fdb-fa02c56d2262 None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:31:50.105053 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69037dab-7a4a-4b43-8fdb-fa02c56d2262 None None] PATH_INFO: `/` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:31:50.107141 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 920/1833] 10.222.0.49 () {54 vars in 921 bytes} [Wed Jun 15 10:31:50 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:31:50.126000 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94668ae2-efad-4c7e-a48d-beca2e2fcd3f None None] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:31:50.127174 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94668ae2-efad-4c7e-a48d-beca2e2fcd3f None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:31:50.127732 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94668ae2-efad-4c7e-a48d-beca2e2fcd3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:31:50.128505 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 914/1834] 10.222.0.49 () {58 vars in 1223 bytes} [Wed Jun 15 10:31:49 2022] DELETE /identity/v3/users/42a3e7682fa34727ac3dc9b6da45ae89 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:31:50.158901 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.auth.core [None req-94668ae2-efad-4c7e-a48d-beca2e2fcd3f None None] MFA Rules not processed for user `dd03612a525644518517069b3fa43522`. Rule list: `[]` (Enabled: `True`). {{(pid=80119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:31:50.161544 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-94668ae2-efad-4c7e-a48d-beca2e2fcd3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:31:50.259115 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 921/1835] 10.222.0.49 () {58 vars in 1018 bytes} [Wed Jun 15 10:31:50 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 10:31:50.275342 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24a0bf4-40a4-4241-ba8d-34316d346871 admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:31:50.276731 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-d24a0bf4-40a4-4241-ba8d-34316d346871 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:31:50.308656 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24a0bf4-40a4-4241-ba8d-34316d346871 admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:31:50.319703 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f10eef2-b597-4dc2-a38c-44125ca1da3e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd03612a525644518517069b3fa43522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '559b3e713f5844b6ae37151d56dd419f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:31:50.319703 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f10eef2-b597-4dc2-a38c-44125ca1da3e service placement] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:31:50.322557 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f10eef2-b597-4dc2-a38c-44125ca1da3e service placement] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:31:50.322557 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f10eef2-b597-4dc2-a38c-44125ca1da3e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:31:50.322752 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94668ae2-efad-4c7e-a48d-beca2e2fcd3f None None] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:31:50.323720 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-94668ae2-efad-4c7e-a48d-beca2e2fcd3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:31:50.332981 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f10eef2-b597-4dc2-a38c-44125ca1da3e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:31:50.334820 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f10eef2-b597-4dc2-a38c-44125ca1da3e service placement] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:31:50.373915 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 915/1836] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Jun 15 10:31:50 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:31:50.444610 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94668ae2-efad-4c7e-a48d-beca2e2fcd3f None None] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:31:50.453689 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9072dbd-84dc-40ec-8dab-f54d5fbe2117 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:31:50.454717 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9072dbd-84dc-40ec-8dab-f54d5fbe2117 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:31:50.455075 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9072dbd-84dc-40ec-8dab-f54d5fbe2117 admin admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:31:50.455497 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9072dbd-84dc-40ec-8dab-f54d5fbe2117 admin admin] PATH_INFO: `/v3/users/318f2b2a3f024c7a8e56269feb343d9f` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:31:50.457620 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9072dbd-84dc-40ec-8dab-f54d5fbe2117 admin admin] RBAC: Authorizing `identity:delete_user(user_id=318f2b2a3f024c7a8e56269feb343d9f)` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:31:50.459839 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9072dbd-84dc-40ec-8dab-f54d5fbe2117 admin admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:31:50.530706 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-f9072dbd-84dc-40ec-8dab-f54d5fbe2117 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '318f2b2a3f024c7a8e56269feb343d9f'} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:31:50.540356 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-f9072dbd-84dc-40ec-8dab-f54d5fbe2117 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '318f2b2a3f024c7a8e56269feb343d9f'} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:31:50.541480 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-f9072dbd-84dc-40ec-8dab-f54d5fbe2117 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '318f2b2a3f024c7a8e56269feb343d9f'} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:31:50.542378 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-f9072dbd-84dc-40ec-8dab-f54d5fbe2117 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '318f2b2a3f024c7a8e56269feb343d9f'} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:31:50.557942 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-f9072dbd-84dc-40ec-8dab-f54d5fbe2117 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '318f2b2a3f024c7a8e56269feb343d9f'} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:31:50.568948 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:DELETE=13,keystone:SELECT=85,keystone:INSERT=2 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:31:50.598141 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 922/1837] 10.222.0.49 () {58 vars in 1223 bytes} [Wed Jun 15 10:31:50 2022] DELETE /identity/v3/users/318f2b2a3f024c7a8e56269feb343d9f => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 10:31:50.745458 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f10eef2-b597-4dc2-a38c-44125ca1da3e service placement] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:31:50.747297 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-3f10eef2-b597-4dc2-a38c-44125ca1da3e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:31:50.765802 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f10eef2-b597-4dc2-a38c-44125ca1da3e service placement] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:31:50.815505 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9072dbd-84dc-40ec-8dab-f54d5fbe2117 admin admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:31:50.817760 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-f9072dbd-84dc-40ec-8dab-f54d5fbe2117 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:31:50.883679 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89ecf52-685e-4e54-ba38-0e9583cdcca7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:31:50.885094 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89ecf52-685e-4e54-ba38-0e9583cdcca7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:31:50.885753 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89ecf52-685e-4e54-ba38-0e9583cdcca7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:31:50.886236 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89ecf52-685e-4e54-ba38-0e9583cdcca7 admin admin] PATH_INFO: `/v3/projects/4150b5d8f39d4f8f81738e8a2c427d48` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:31:50.888542 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89ecf52-685e-4e54-ba38-0e9583cdcca7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4150b5d8f39d4f8f81738e8a2c427d48)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:31:50.891263 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89ecf52-685e-4e54-ba38-0e9583cdcca7 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:31:50.950081 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:31:50.957557 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9072dbd-84dc-40ec-8dab-f54d5fbe2117 admin admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:31:50.997322 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b61cbc6-22b3-4072-bc8c-2294f1f1d9ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:31:51.001198 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b61cbc6-22b3-4072-bc8c-2294f1f1d9ec admin admin] REQUEST_METHOD: `DELETE` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:31:51.001494 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b61cbc6-22b3-4072-bc8c-2294f1f1d9ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:31:51.002206 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b61cbc6-22b3-4072-bc8c-2294f1f1d9ec admin admin] PATH_INFO: `/v3/projects/0e883186e4784ba4a94a7b7f649b6d43` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:31:51.012003 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b61cbc6-22b3-4072-bc8c-2294f1f1d9ec admin admin] RBAC: Authorizing `identity:delete_project(project_id=0e883186e4784ba4a94a7b7f649b6d43)` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:31:51.013309 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-b89ecf52-685e-4e54-ba38-0e9583cdcca7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4150b5d8f39d4f8f81738e8a2c427d48', 'request_id': 'req-b89ecf52-685e-4e54-ba38-0e9583cdcca7'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:31:51.018379 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b61cbc6-22b3-4072-bc8c-2294f1f1d9ec admin admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:31:51.036746 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-b89ecf52-685e-4e54-ba38-0e9583cdcca7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4150b5d8f39d4f8f81738e8a2c427d48', 'request_id': 'req-b89ecf52-685e-4e54-ba38-0e9583cdcca7'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:31:51.050603 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-b89ecf52-685e-4e54-ba38-0e9583cdcca7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4150b5d8f39d4f8f81738e8a2c427d48', 'request_id': 'req-b89ecf52-685e-4e54-ba38-0e9583cdcca7'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:31:51.087398 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-b89ecf52-685e-4e54-ba38-0e9583cdcca7 admin admin] The token cache is being invalidate because project 4150b5d8f39d4f8f81738e8a2c427d48 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80120) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 10:31:51.088404 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-b89ecf52-685e-4e54-ba38-0e9583cdcca7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:31:51.090315 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-b89ecf52-685e-4e54-ba38-0e9583cdcca7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:31:51.095106 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 916/1838] 10.222.0.49 () {58 vars in 1232 bytes} [Wed Jun 15 10:31:50 2022] DELETE /identity/v3/projects/4150b5d8f39d4f8f81738e8a2c427d48 => generated 0 bytes in 350 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:31:51.100157 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cb96f2-a06c-403b-9cbe-220d48a08744 None None] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:31:51.101795 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cb96f2-a06c-403b-9cbe-220d48a08744 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:31:51.102043 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cb96f2-a06c-403b-9cbe-220d48a08744 None None] PATH_INFO: `/` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:31:51.104326 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 917/1839] 10.222.0.49 () {54 vars in 921 bytes} [Wed Jun 15 10:31:51 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:31:51.105713 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72d13ed-124a-4377-a1d8-2af6edd95a12 None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:31:51.108495 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72d13ed-124a-4377-a1d8-2af6edd95a12 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:31:51.110484 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72d13ed-124a-4377-a1d8-2af6edd95a12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:31:51.148711 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-c72d13ed-124a-4377-a1d8-2af6edd95a12 None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:31:51.151176 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-c72d13ed-124a-4377-a1d8-2af6edd95a12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:31:51.169162 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-4b61cbc6-22b3-4072-bc8c-2294f1f1d9ec admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0e883186e4784ba4a94a7b7f649b6d43', 'request_id': 'req-4b61cbc6-22b3-4072-bc8c-2294f1f1d9ec'} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:31:51.184061 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-4b61cbc6-22b3-4072-bc8c-2294f1f1d9ec admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0e883186e4784ba4a94a7b7f649b6d43', 'request_id': 'req-4b61cbc6-22b3-4072-bc8c-2294f1f1d9ec'} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:31:51.201256 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-4b61cbc6-22b3-4072-bc8c-2294f1f1d9ec admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0e883186e4784ba4a94a7b7f649b6d43', 'request_id': 'req-4b61cbc6-22b3-4072-bc8c-2294f1f1d9ec'} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:31:51.237159 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-4b61cbc6-22b3-4072-bc8c-2294f1f1d9ec admin admin] The token cache is being invalidate because project 0e883186e4784ba4a94a7b7f649b6d43 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80119) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 10:31:51.237599 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-4b61cbc6-22b3-4072-bc8c-2294f1f1d9ec admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:31:51.239346 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-4b61cbc6-22b3-4072-bc8c-2294f1f1d9ec admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:31:51.243465 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 923/1840] 10.222.0.49 () {58 vars in 1232 bytes} [Wed Jun 15 10:31:50 2022] DELETE /identity/v3/projects/0e883186e4784ba4a94a7b7f649b6d43 => generated 0 bytes in 426 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 10:31:51.246341 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6b3337-ac38-49a2-8de1-cb4d6e638de5 None None] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:31:51.247838 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6b3337-ac38-49a2-8de1-cb4d6e638de5 None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:31:51.249512 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6b3337-ac38-49a2-8de1-cb4d6e638de5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:31:51.283448 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.auth.core [None req-9a6b3337-ac38-49a2-8de1-cb4d6e638de5 None None] MFA Rules not processed for user `5ac54528370d4747b6e8777fba9d0c82`. Rule list: `[]` (Enabled: `True`). {{(pid=80119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:31:51.284854 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-9a6b3337-ac38-49a2-8de1-cb4d6e638de5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:31:51.325590 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 918/1841] 10.222.0.49 () {58 vars in 957 bytes} [Wed Jun 15 10:31:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:31:51.328908 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72d13ed-124a-4377-a1d8-2af6edd95a12 None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:31:51.332632 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-c72d13ed-124a-4377-a1d8-2af6edd95a12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:31:51.365063 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72d13ed-124a-4377-a1d8-2af6edd95a12 None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:31:51.385665 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71626d5b-dfdb-47fb-98bc-c498e3b6f9b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:31:51.387072 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71626d5b-dfdb-47fb-98bc-c498e3b6f9b8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:31:51.388202 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71626d5b-dfdb-47fb-98bc-c498e3b6f9b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:31:51.388507 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71626d5b-dfdb-47fb-98bc-c498e3b6f9b8 admin admin] PATH_INFO: `/v3/users/2fc5998233d146a3ba9701410af7bc58` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:31:51.393322 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71626d5b-dfdb-47fb-98bc-c498e3b6f9b8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2fc5998233d146a3ba9701410af7bc58)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:31:51.399278 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71626d5b-dfdb-47fb-98bc-c498e3b6f9b8 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:31:51.457272 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 924/1842] 10.222.0.49 () {58 vars in 1018 bytes} [Wed Jun 15 10:31:51 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:31:51.459622 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ad86af-c725-4e82-9694-4464a2338778 None None] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:31:51.459622 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ad86af-c725-4e82-9694-4464a2338778 None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:31:51.459622 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ad86af-c725-4e82-9694-4464a2338778 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:31:51.513445 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-71626d5b-dfdb-47fb-98bc-c498e3b6f9b8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2fc5998233d146a3ba9701410af7bc58'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:31:51.523407 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.auth.core [None req-72ad86af-c725-4e82-9694-4464a2338778 None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:31:51.526588 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-72ad86af-c725-4e82-9694-4464a2338778 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:31:51.528627 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-71626d5b-dfdb-47fb-98bc-c498e3b6f9b8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2fc5998233d146a3ba9701410af7bc58'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:31:51.529675 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-71626d5b-dfdb-47fb-98bc-c498e3b6f9b8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2fc5998233d146a3ba9701410af7bc58'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:31:51.531764 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-71626d5b-dfdb-47fb-98bc-c498e3b6f9b8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2fc5998233d146a3ba9701410af7bc58'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:31:51.546555 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 925/1843] 10.222.0.49 () {58 vars in 957 bytes} [Wed Jun 15 10:31:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:31:51.548804 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ad86af-c725-4e82-9694-4464a2338778 None None] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:31:51.550910 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-72ad86af-c725-4e82-9694-4464a2338778 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:31:51.554522 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-71626d5b-dfdb-47fb-98bc-c498e3b6f9b8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2fc5998233d146a3ba9701410af7bc58'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:31:51.587356 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:31:51.596204 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ad86af-c725-4e82-9694-4464a2338778 None None] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:31:51.596957 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 919/1844] 10.222.0.49 () {58 vars in 1223 bytes} [Wed Jun 15 10:31:51 2022] DELETE /identity/v3/users/2fc5998233d146a3ba9701410af7bc58 => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:31:51.598495 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71626d5b-dfdb-47fb-98bc-c498e3b6f9b8 admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:31:51.640945 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:31:51.712550 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71626d5b-dfdb-47fb-98bc-c498e3b6f9b8 admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:31:51.718416 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f3467e-7b15-494c-b414-4bcf2d3833eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:31:51.719339 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f3467e-7b15-494c-b414-4bcf2d3833eb admin admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:31:51.719703 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f3467e-7b15-494c-b414-4bcf2d3833eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:31:51.720057 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f3467e-7b15-494c-b414-4bcf2d3833eb admin admin] PATH_INFO: `/v3/domains` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:31:51.720976 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019bcef3-cb1a-4433-a51f-4a9d11661093 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ac54528370d4747b6e8777fba9d0c82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '559b3e713f5844b6ae37151d56dd419f', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:31:51.722407 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019bcef3-cb1a-4433-a51f-4a9d11661093 service nova] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:31:51.722705 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019bcef3-cb1a-4433-a51f-4a9d11661093 service nova] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:31:51.722988 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019bcef3-cb1a-4433-a51f-4a9d11661093 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:31:51.725090 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-019bcef3-cb1a-4433-a51f-4a9d11661093 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:31:51.731051 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f3467e-7b15-494c-b414-4bcf2d3833eb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:31:51.731509 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f3467e-7b15-494c-b414-4bcf2d3833eb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:31:51.742182 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-019bcef3-cb1a-4433-a51f-4a9d11661093 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:31:51.744963 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-019bcef3-cb1a-4433-a51f-4a9d11661093 service nova] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:31:51.745269 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f3467e-7b15-494c-b414-4bcf2d3833eb admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:31:51.759407 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 926/1845] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Jun 15 10:31:51 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 212 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:31:51.793172 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 920/1846] 10.222.0.49 () {58 vars in 1152 bytes} [Wed Jun 15 10:31:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:31:51.795079 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019bcef3-cb1a-4433-a51f-4a9d11661093 service nova] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:31:51.832863 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019bcef3-cb1a-4433-a51f-4a9d11661093 service nova] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:31:51.841217 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471d9653-6cae-45d5-9f5e-2f0e2d8509d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:31:51.843420 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471d9653-6cae-45d5-9f5e-2f0e2d8509d4 admin admin] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:31:51.843914 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471d9653-6cae-45d5-9f5e-2f0e2d8509d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:31:51.844086 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471d9653-6cae-45d5-9f5e-2f0e2d8509d4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:31:51.844844 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471d9653-6cae-45d5-9f5e-2f0e2d8509d4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:31:51.848727 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471d9653-6cae-45d5-9f5e-2f0e2d8509d4 admin admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:31:51.902778 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 927/1847] 10.222.0.49 () {60 vars in 1152 bytes} [Wed Jun 15 10:31:51 2022] POST /identity/v3/projects => generated 373 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 10:31:51.906748 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f3467e-7b15-494c-b414-4bcf2d3833eb admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:31:51.921081 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f3467e-7b15-494c-b414-4bcf2d3833eb admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:31:52.042997 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d70718-3155-4b8a-b510-b8b624a45922 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:31:52.043846 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d70718-3155-4b8a-b510-b8b624a45922 admin admin] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:31:52.044219 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d70718-3155-4b8a-b510-b8b624a45922 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:31:52.044551 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d70718-3155-4b8a-b510-b8b624a45922 admin admin] PATH_INFO: `/v3/users` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:31:52.045405 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d70718-3155-4b8a-b510-b8b624a45922 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:31:52.048890 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d70718-3155-4b8a-b510-b8b624a45922 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:31:52.089778 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471d9653-6cae-45d5-9f5e-2f0e2d8509d4 admin admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:31:52.092706 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-471d9653-6cae-45d5-9f5e-2f0e2d8509d4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:31:52.106514 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 921/1848] 10.222.0.49 () {60 vars in 1143 bytes} [Wed Jun 15 10:31:51 2022] POST /identity/v3/users => generated 287 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 10:31:52.109350 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d70718-3155-4b8a-b510-b8b624a45922 admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:31:52.125352 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471d9653-6cae-45d5-9f5e-2f0e2d8509d4 admin admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:31:52.140973 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d70718-3155-4b8a-b510-b8b624a45922 admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:31:52.141854 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a8389b-0a9c-475c-a283-9b34c717bdaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:31:52.145093 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a8389b-0a9c-475c-a283-9b34c717bdaf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:31:52.147610 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a8389b-0a9c-475c-a283-9b34c717bdaf admin admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:31:52.149497 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a8389b-0a9c-475c-a283-9b34c717bdaf admin admin] PATH_INFO: `/v3/projects/925619655cc54cc29e408533cec49d9e` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:31:52.153153 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a8389b-0a9c-475c-a283-9b34c717bdaf admin admin] RBAC: Authorizing `identity:delete_project(project_id=925619655cc54cc29e408533cec49d9e)` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:31:52.157004 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850f410e-3eb7-462d-b3d6-1773cfb27a31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:31:52.160975 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a8389b-0a9c-475c-a283-9b34c717bdaf admin admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:31:52.161675 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850f410e-3eb7-462d-b3d6-1773cfb27a31 admin admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:31:52.161979 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850f410e-3eb7-462d-b3d6-1773cfb27a31 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:31:52.163480 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850f410e-3eb7-462d-b3d6-1773cfb27a31 admin admin] PATH_INFO: `/v3/roles` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:31:52.165718 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-850f410e-3eb7-462d-b3d6-1773cfb27a31 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:31:52.176422 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-850f410e-3eb7-462d-b3d6-1773cfb27a31 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:31:52.193067 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 922/1849] 10.222.0.49 () {58 vars in 1121 bytes} [Wed Jun 15 10:31:52 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 10:31:52.197503 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850f410e-3eb7-462d-b3d6-1773cfb27a31 admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:31:52.220872 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850f410e-3eb7-462d-b3d6-1773cfb27a31 admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:31:52.230262 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b27b8c6-fe63-48f1-afca-fcf1e296432a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:31:52.230964 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b27b8c6-fe63-48f1-afca-fcf1e296432a admin admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:31:52.234031 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b27b8c6-fe63-48f1-afca-fcf1e296432a admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:31:52.234232 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b27b8c6-fe63-48f1-afca-fcf1e296432a admin admin] PATH_INFO: `/v3/roles` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:31:52.235233 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b27b8c6-fe63-48f1-afca-fcf1e296432a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:31:52.240797 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b27b8c6-fe63-48f1-afca-fcf1e296432a admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:31:52.277180 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 923/1850] 10.222.0.49 () {58 vars in 1121 bytes} [Wed Jun 15 10:31:52 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 10:31:52.280173 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b27b8c6-fe63-48f1-afca-fcf1e296432a admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:31:52.306419 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b27b8c6-fe63-48f1-afca-fcf1e296432a admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:31:52.327113 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4e9cb2-a2cb-4bcf-a2a2-0b7de1f7b959 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:31:52.329455 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4e9cb2-a2cb-4bcf-a2a2-0b7de1f7b959 admin admin] REQUEST_METHOD: `PUT` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:31:52.329676 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4e9cb2-a2cb-4bcf-a2a2-0b7de1f7b959 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:31:52.329960 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4e9cb2-a2cb-4bcf-a2a2-0b7de1f7b959 admin admin] PATH_INFO: `/v3/projects/5c0b3805b6d74092ac6710bb6b00cb1e/users/d18e5e85b8be4225af127b55b6bea9f1/roles/6876e57cc51c448097ab109d35b2a3da` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:31:52.350974 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-d4a8389b-0a9c-475c-a283-9b34c717bdaf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '925619655cc54cc29e408533cec49d9e', 'request_id': 'req-d4a8389b-0a9c-475c-a283-9b34c717bdaf'} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:31:52.372648 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-d4a8389b-0a9c-475c-a283-9b34c717bdaf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '925619655cc54cc29e408533cec49d9e', 'request_id': 'req-d4a8389b-0a9c-475c-a283-9b34c717bdaf'} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:31:52.376341 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c4e9cb2-a2cb-4bcf-a2a2-0b7de1f7b959 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5c0b3805b6d74092ac6710bb6b00cb1e, user_id=d18e5e85b8be4225af127b55b6bea9f1, role_id=6876e57cc51c448097ab109d35b2a3da)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:31:52.384390 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-d4a8389b-0a9c-475c-a283-9b34c717bdaf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '925619655cc54cc29e408533cec49d9e', 'request_id': 'req-d4a8389b-0a9c-475c-a283-9b34c717bdaf'} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:31:52.388423 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c4e9cb2-a2cb-4bcf-a2a2-0b7de1f7b959 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:31:52.421420 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-d4a8389b-0a9c-475c-a283-9b34c717bdaf admin admin] The token cache is being invalidate because project 925619655cc54cc29e408533cec49d9e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80119) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 10:31:52.425252 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-d4a8389b-0a9c-475c-a283-9b34c717bdaf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:31:52.426690 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-d4a8389b-0a9c-475c-a283-9b34c717bdaf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:31:52.435830 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 928/1851] 10.222.0.49 () {58 vars in 1232 bytes} [Wed Jun 15 10:31:52 2022] DELETE /identity/v3/projects/925619655cc54cc29e408533cec49d9e => generated 0 bytes in 343 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:31:52.436027 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 924/1852] 10.222.0.49 () {60 vars in 1482 bytes} [Wed Jun 15 10:31:52 2022] PUT /identity/v3/projects/5c0b3805b6d74092ac6710bb6b00cb1e/users/d18e5e85b8be4225af127b55b6bea9f1/roles/6876e57cc51c448097ab109d35b2a3da => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:31:52.447857 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf7f355-51b9-4f0c-80f3-df6c3871a3d8 None None] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:31:52.448151 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf7f355-51b9-4f0c-80f3-df6c3871a3d8 None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:31:52.448816 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf7f355-51b9-4f0c-80f3-df6c3871a3d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:31:52.449887 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9cd3aa-8bb6-4c30-9fd9-5af670ea0234 None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:31:52.450181 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9cd3aa-8bb6-4c30-9fd9-5af670ea0234 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:31:52.450460 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9cd3aa-8bb6-4c30-9fd9-5af670ea0234 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:31:52.497916 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-0b9cd3aa-8bb6-4c30-9fd9-5af670ea0234 None None] MFA Rules not processed for user `d18e5e85b8be4225af127b55b6bea9f1`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:31:52.499788 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-0b9cd3aa-8bb6-4c30-9fd9-5af670ea0234 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:31:52.526522 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.auth.core [None req-0cf7f355-51b9-4f0c-80f3-df6c3871a3d8 None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:31:52.530426 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-0cf7f355-51b9-4f0c-80f3-df6c3871a3d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:31:52.597764 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:31:52.630394 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 925/1853] 10.222.0.49 () {58 vars in 957 bytes} [Wed Jun 15 10:31:52 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:31:52.637251 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9cd3aa-8bb6-4c30-9fd9-5af670ea0234 None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:31:52.638787 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-0b9cd3aa-8bb6-4c30-9fd9-5af670ea0234 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:31:52.657207 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:31:52.696298 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 929/1854] 10.222.0.49 () {58 vars in 957 bytes} [Wed Jun 15 10:31:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 251 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:31:52.697630 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf7f355-51b9-4f0c-80f3-df6c3871a3d8 None None] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:31:52.699397 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-0cf7f355-51b9-4f0c-80f3-df6c3871a3d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:31:52.716386 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf7f355-51b9-4f0c-80f3-df6c3871a3d8 None None] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:31:52.730874 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e8a0b8-bf06-4511-9917-36d89189efe0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:31:52.732353 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e8a0b8-bf06-4511-9917-36d89189efe0 admin admin] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:31:52.732973 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e8a0b8-bf06-4511-9917-36d89189efe0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:31:52.733456 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e8a0b8-bf06-4511-9917-36d89189efe0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:31:52.734885 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e8a0b8-bf06-4511-9917-36d89189efe0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:31:52.738936 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e8a0b8-bf06-4511-9917-36d89189efe0 admin admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:31:52.826073 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 930/1855] 10.222.0.49 () {60 vars in 1152 bytes} [Wed Jun 15 10:31:52 2022] POST /identity/v3/projects => generated 369 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 10:31:52.827393 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a562f44-e82e-4287-820b-66e4d8647023 None None] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:31:52.827687 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a562f44-e82e-4287-820b-66e4d8647023 None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:31:52.827965 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a562f44-e82e-4287-820b-66e4d8647023 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:31:52.857700 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9cd3aa-8bb6-4c30-9fd9-5af670ea0234 None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:31:52.866954 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.auth.core [None req-7a562f44-e82e-4287-820b-66e4d8647023 None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:31:52.868780 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-7a562f44-e82e-4287-820b-66e4d8647023 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:31:52.984134 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9910271-5bdf-4e45-9034-a86b058a33d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:31:52.985087 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9910271-5bdf-4e45-9034-a86b058a33d2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:31:52.985372 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9910271-5bdf-4e45-9034-a86b058a33d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:31:52.985653 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9910271-5bdf-4e45-9034-a86b058a33d2 admin admin] PATH_INFO: `/v3/users/68ab1e11148e413386dd8aa4aefae2ce` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:31:52.988231 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 931/1856] 10.222.0.49 () {58 vars in 957 bytes} [Wed Jun 15 10:31:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:31:52.989173 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a562f44-e82e-4287-820b-66e4d8647023 None None] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:31:52.989733 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9910271-5bdf-4e45-9034-a86b058a33d2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=68ab1e11148e413386dd8aa4aefae2ce)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:31:52.991776 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9910271-5bdf-4e45-9034-a86b058a33d2 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:31:53.003892 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a562f44-e82e-4287-820b-66e4d8647023 None None] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:31:53.012788 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d2a05b-814f-4e8c-8ab5-69b56eb3286b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:31:53.013556 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d2a05b-814f-4e8c-8ab5-69b56eb3286b admin admin] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:31:53.013844 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d2a05b-814f-4e8c-8ab5-69b56eb3286b admin admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:31:53.014143 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d2a05b-814f-4e8c-8ab5-69b56eb3286b admin admin] PATH_INFO: `/v3/users` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:31:53.014825 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d2a05b-814f-4e8c-8ab5-69b56eb3286b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:31:53.019258 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d2a05b-814f-4e8c-8ab5-69b56eb3286b admin admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:31:53.069040 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 932/1857] 10.222.0.49 () {60 vars in 1143 bytes} [Wed Jun 15 10:31:52 2022] POST /identity/v3/users => generated 291 bytes in 80 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 10:31:53.071820 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d2a05b-814f-4e8c-8ab5-69b56eb3286b admin admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:31:53.093534 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d2a05b-814f-4e8c-8ab5-69b56eb3286b admin admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:31:53.108543 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-e9910271-5bdf-4e45-9034-a86b058a33d2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '68ab1e11148e413386dd8aa4aefae2ce'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:31:53.112318 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5f8c14-6a49-4c9f-b8cb-bd72019325a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:31:53.113061 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5f8c14-6a49-4c9f-b8cb-bd72019325a2 admin admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:31:53.113348 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5f8c14-6a49-4c9f-b8cb-bd72019325a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:31:53.113627 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5f8c14-6a49-4c9f-b8cb-bd72019325a2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:31:53.114571 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b5f8c14-6a49-4c9f-b8cb-bd72019325a2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:31:53.114924 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b5f8c14-6a49-4c9f-b8cb-bd72019325a2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:31:53.117143 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b5f8c14-6a49-4c9f-b8cb-bd72019325a2 admin admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:31:53.129569 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-e9910271-5bdf-4e45-9034-a86b058a33d2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '68ab1e11148e413386dd8aa4aefae2ce'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:31:53.131562 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-e9910271-5bdf-4e45-9034-a86b058a33d2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '68ab1e11148e413386dd8aa4aefae2ce'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:31:53.133408 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-e9910271-5bdf-4e45-9034-a86b058a33d2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '68ab1e11148e413386dd8aa4aefae2ce'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:31:53.138152 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 933/1858] 10.222.0.49 () {58 vars in 1152 bytes} [Wed Jun 15 10:31:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:31:53.139487 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5f8c14-6a49-4c9f-b8cb-bd72019325a2 admin admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:31:53.141377 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-1b5f8c14-6a49-4c9f-b8cb-bd72019325a2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:31:53.151332 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-e9910271-5bdf-4e45-9034-a86b058a33d2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '68ab1e11148e413386dd8aa4aefae2ce'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:31:53.159070 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5f8c14-6a49-4c9f-b8cb-bd72019325a2 admin admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:31:53.166344 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17e3706-6cd0-4261-8187-64b386c2944b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:31:53.166912 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17e3706-6cd0-4261-8187-64b386c2944b admin admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:31:53.167225 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17e3706-6cd0-4261-8187-64b386c2944b admin admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:31:53.167507 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17e3706-6cd0-4261-8187-64b386c2944b admin admin] PATH_INFO: `/v3/roles` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:31:53.168620 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e17e3706-6cd0-4261-8187-64b386c2944b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:31:53.171063 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e17e3706-6cd0-4261-8187-64b386c2944b admin admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:31:53.185675 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 934/1859] 10.222.0.49 () {58 vars in 1121 bytes} [Wed Jun 15 10:31:53 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 10:31:53.186823 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17e3706-6cd0-4261-8187-64b386c2944b admin admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:31:53.189278 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 926/1860] 10.222.0.49 () {58 vars in 1223 bytes} [Wed Jun 15 10:31:52 2022] DELETE /identity/v3/users/68ab1e11148e413386dd8aa4aefae2ce => generated 0 bytes in 559 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:31:53.190250 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9910271-5bdf-4e45-9034-a86b058a33d2 admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:31:53.193870 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-e17e3706-6cd0-4261-8187-64b386c2944b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:31:53.210508 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17e3706-6cd0-4261-8187-64b386c2944b admin admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:31:53.221005 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:31:53.324694 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0515c85-2189-4f78-a1bc-0adc4acbfe9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:31:53.325712 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0515c85-2189-4f78-a1bc-0adc4acbfe9d admin admin] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:31:53.325974 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0515c85-2189-4f78-a1bc-0adc4acbfe9d admin admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:31:53.326336 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0515c85-2189-4f78-a1bc-0adc4acbfe9d admin admin] PATH_INFO: `/v3/projects` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:31:53.327340 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0515c85-2189-4f78-a1bc-0adc4acbfe9d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:31:53.329605 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0515c85-2189-4f78-a1bc-0adc4acbfe9d admin admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:31:53.343468 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9910271-5bdf-4e45-9034-a86b058a33d2 admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:31:53.359062 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f960f1cc-4944-45bd-9dea-652987e9cff3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:31:53.360032 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f960f1cc-4944-45bd-9dea-652987e9cff3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:31:53.360320 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f960f1cc-4944-45bd-9dea-652987e9cff3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:31:53.360616 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f960f1cc-4944-45bd-9dea-652987e9cff3 admin admin] PATH_INFO: `/v3/projects/974ed5ca2296425fbc2cea2aee4c3ca3/users/4eee6512e3cf47bf989a0b9a8bebb088/roles/8051174e7a7947d5b991ebd793c2e543` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:31:53.386393 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 935/1861] 10.222.0.49 () {60 vars in 1152 bytes} [Wed Jun 15 10:31:53 2022] POST /identity/v3/projects => generated 387 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 10:31:53.386393 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5435a47e-b02f-4dbf-b1e9-596d35dcd737 None None] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:31:53.386393 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5435a47e-b02f-4dbf-b1e9-596d35dcd737 None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:31:53.386393 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5435a47e-b02f-4dbf-b1e9-596d35dcd737 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:31:53.401378 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f960f1cc-4944-45bd-9dea-652987e9cff3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=974ed5ca2296425fbc2cea2aee4c3ca3, user_id=4eee6512e3cf47bf989a0b9a8bebb088, role_id=8051174e7a7947d5b991ebd793c2e543)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:31:53.407181 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f960f1cc-4944-45bd-9dea-652987e9cff3 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:31:53.428420 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.auth.core [None req-5435a47e-b02f-4dbf-b1e9-596d35dcd737 None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:31:53.430552 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-5435a47e-b02f-4dbf-b1e9-596d35dcd737 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:31:53.434078 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 927/1862] 10.222.0.49 () {60 vars in 1482 bytes} [Wed Jun 15 10:31:53 2022] PUT /identity/v3/projects/974ed5ca2296425fbc2cea2aee4c3ca3/users/4eee6512e3cf47bf989a0b9a8bebb088/roles/8051174e7a7947d5b991ebd793c2e543 => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:31:53.434927 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f960f1cc-4944-45bd-9dea-652987e9cff3 admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:31:53.448748 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:31:53.538797 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 936/1863] 10.222.0.49 () {58 vars in 957 bytes} [Wed Jun 15 10:31:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:31:53.543914 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5435a47e-b02f-4dbf-b1e9-596d35dcd737 None None] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:31:53.554132 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5435a47e-b02f-4dbf-b1e9-596d35dcd737 None None] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:31:53.565929 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5f7395-33e5-492b-9ee4-961d679b5663 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:31:53.566950 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5f7395-33e5-492b-9ee4-961d679b5663 admin admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:31:53.567291 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5f7395-33e5-492b-9ee4-961d679b5663 admin admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:31:53.567570 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5f7395-33e5-492b-9ee4-961d679b5663 admin admin] PATH_INFO: `/v3/roles` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:31:53.568584 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c5f7395-33e5-492b-9ee4-961d679b5663 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:31:53.571391 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c5f7395-33e5-492b-9ee4-961d679b5663 admin admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:31:53.588910 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f960f1cc-4944-45bd-9dea-652987e9cff3 admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:31:53.589570 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 937/1864] 10.222.0.49 () {58 vars in 1121 bytes} [Wed Jun 15 10:31:53 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 10:31:53.594065 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5f7395-33e5-492b-9ee4-961d679b5663 admin admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:31:53.595860 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-5c5f7395-33e5-492b-9ee4-961d679b5663 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:31:53.603914 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b710c90c-220e-4fe5-bab5-5e384c6a3af0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:31:53.604992 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b710c90c-220e-4fe5-bab5-5e384c6a3af0 admin admin] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:31:53.605482 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b710c90c-220e-4fe5-bab5-5e384c6a3af0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:31:53.605960 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b710c90c-220e-4fe5-bab5-5e384c6a3af0 admin admin] PATH_INFO: `/v3/users` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:31:53.607250 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b710c90c-220e-4fe5-bab5-5e384c6a3af0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:31:53.610596 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b710c90c-220e-4fe5-bab5-5e384c6a3af0 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:31:53.644725 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 928/1865] 10.222.0.49 () {60 vars in 1143 bytes} [Wed Jun 15 10:31:53 2022] POST /identity/v3/users => generated 294 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 10:31:53.646161 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b710c90c-220e-4fe5-bab5-5e384c6a3af0 admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:31:53.667584 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b710c90c-220e-4fe5-bab5-5e384c6a3af0 admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:31:53.682672 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8537f458-00e4-4ed7-980d-bef1774deb95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:31:53.683632 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8537f458-00e4-4ed7-980d-bef1774deb95 admin admin] REQUEST_METHOD: `PUT` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:31:53.683911 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8537f458-00e4-4ed7-980d-bef1774deb95 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:31:53.684176 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8537f458-00e4-4ed7-980d-bef1774deb95 admin admin] PATH_INFO: `/v3/projects/974ed5ca2296425fbc2cea2aee4c3ca3/users/4eee6512e3cf47bf989a0b9a8bebb088/roles/8051174e7a7947d5b991ebd793c2e543` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:31:53.687055 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8537f458-00e4-4ed7-980d-bef1774deb95 admin admin] RBAC: Authorizing `identity:create_grant(project_id=974ed5ca2296425fbc2cea2aee4c3ca3, user_id=4eee6512e3cf47bf989a0b9a8bebb088, role_id=8051174e7a7947d5b991ebd793c2e543)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:31:53.689726 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8537f458-00e4-4ed7-980d-bef1774deb95 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:31:53.711646 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5f7395-33e5-492b-9ee4-961d679b5663 admin admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:31:53.712522 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 929/1866] 10.222.0.49 () {60 vars in 1482 bytes} [Wed Jun 15 10:31:53 2022] PUT /identity/v3/projects/974ed5ca2296425fbc2cea2aee4c3ca3/users/4eee6512e3cf47bf989a0b9a8bebb088/roles/8051174e7a7947d5b991ebd793c2e543 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:31:53.715975 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8537f458-00e4-4ed7-980d-bef1774deb95 admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:31:53.810353 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6847f3c-9e41-41b1-9cf5-2650adb8fb33 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0cb44a17d205431793e9a8791e477a32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '559b3e713f5844b6ae37151d56dd419f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:31:53.811869 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6847f3c-9e41-41b1-9cf5-2650adb8fb33 service neutron] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:31:53.812165 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6847f3c-9e41-41b1-9cf5-2650adb8fb33 service neutron] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:31:53.814146 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6847f3c-9e41-41b1-9cf5-2650adb8fb33 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:31:53.866591 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8537f458-00e4-4ed7-980d-bef1774deb95 admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:31:53.881033 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909f7551-00f4-4012-99b6-4e04e4362aed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:31:53.881897 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909f7551-00f4-4012-99b6-4e04e4362aed admin admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:31:53.882007 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909f7551-00f4-4012-99b6-4e04e4362aed admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:31:53.882289 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909f7551-00f4-4012-99b6-4e04e4362aed admin admin] PATH_INFO: `/v3/roles` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:31:53.883111 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909f7551-00f4-4012-99b6-4e04e4362aed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:31:53.885223 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909f7551-00f4-4012-99b6-4e04e4362aed admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:31:53.920188 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 930/1867] 10.222.0.49 () {58 vars in 1121 bytes} [Wed Jun 15 10:31:53 2022] GET /identity/v3/roles => generated 1420 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 10:31:53.920188 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc7dd4c-8ba4-4d65-9a0a-7cffca256d08 None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:31:53.920188 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc7dd4c-8ba4-4d65-9a0a-7cffca256d08 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:31:53.920188 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc7dd4c-8ba4-4d65-9a0a-7cffca256d08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:31:53.948475 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:31:53.953110 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6847f3c-9e41-41b1-9cf5-2650adb8fb33 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:31:53.954644 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6847f3c-9e41-41b1-9cf5-2650adb8fb33 service neutron] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:31:53.981802 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-3cc7dd4c-8ba4-4d65-9a0a-7cffca256d08 None None] MFA Rules not processed for user `4eee6512e3cf47bf989a0b9a8bebb088`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:31:53.983076 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-3cc7dd4c-8ba4-4d65-9a0a-7cffca256d08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:31:53.988733 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 938/1868] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Jun 15 10:31:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:31:53.989722 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6847f3c-9e41-41b1-9cf5-2650adb8fb33 service neutron] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:31:54.015702 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6847f3c-9e41-41b1-9cf5-2650adb8fb33 service neutron] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:31:54.029757 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542c5f95-accd-4328-89cb-2cdf113728bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:31:54.030456 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542c5f95-accd-4328-89cb-2cdf113728bd admin admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:31:54.030742 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542c5f95-accd-4328-89cb-2cdf113728bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:31:54.031321 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542c5f95-accd-4328-89cb-2cdf113728bd admin admin] PATH_INFO: `/v3/roles` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:31:54.032222 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542c5f95-accd-4328-89cb-2cdf113728bd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:31:54.037072 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542c5f95-accd-4328-89cb-2cdf113728bd admin admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:31:54.044008 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=100 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:31:54.082957 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 939/1869] 10.222.0.49 () {58 vars in 1121 bytes} [Wed Jun 15 10:31:53 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 10:31:54.094157 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542c5f95-accd-4328-89cb-2cdf113728bd admin admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:31:54.125525 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542c5f95-accd-4328-89cb-2cdf113728bd admin admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:31:54.139875 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ea4a1d-3e16-49fc-b0ee-9c54988ebdaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:31:54.141612 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ea4a1d-3e16-49fc-b0ee-9c54988ebdaf admin admin] REQUEST_METHOD: `PUT` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:31:54.141917 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ea4a1d-3e16-49fc-b0ee-9c54988ebdaf admin admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:31:54.142204 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ea4a1d-3e16-49fc-b0ee-9c54988ebdaf admin admin] PATH_INFO: `/v3/projects/f758049b6de94c10a3dafee0083ba61b/users/c762e4c31586473eb81ab8d9298600f6/roles/6876e57cc51c448097ab109d35b2a3da` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:31:54.186202 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ea4a1d-3e16-49fc-b0ee-9c54988ebdaf admin admin] RBAC: Authorizing `identity:create_grant(project_id=f758049b6de94c10a3dafee0083ba61b, user_id=c762e4c31586473eb81ab8d9298600f6, role_id=6876e57cc51c448097ab109d35b2a3da)` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:31:54.194035 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 931/1870] 10.222.0.49 () {58 vars in 957 bytes} [Wed Jun 15 10:31:53 2022] POST /identity/v3/auth/tokens => generated 3303 bytes in 276 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:31:54.195013 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc7dd4c-8ba4-4d65-9a0a-7cffca256d08 None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:31:54.195790 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ea4a1d-3e16-49fc-b0ee-9c54988ebdaf admin admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:31:54.197671 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-3cc7dd4c-8ba4-4d65-9a0a-7cffca256d08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:31:54.234934 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 940/1871] 10.222.0.49 () {60 vars in 1482 bytes} [Wed Jun 15 10:31:54 2022] PUT /identity/v3/projects/f758049b6de94c10a3dafee0083ba61b/users/c762e4c31586473eb81ab8d9298600f6/roles/6876e57cc51c448097ab109d35b2a3da => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:31:54.234934 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ea4a1d-3e16-49fc-b0ee-9c54988ebdaf admin admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:31:54.237325 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-d8ea4a1d-3e16-49fc-b0ee-9c54988ebdaf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:31:54.250207 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc7dd4c-8ba4-4d65-9a0a-7cffca256d08 None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:31:54.330206 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63b1e79-8930-4ef0-817f-5c8305d69b77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:31:54.330916 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63b1e79-8930-4ef0-817f-5c8305d69b77 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:31:54.331233 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63b1e79-8930-4ef0-817f-5c8305d69b77 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:31:54.331511 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63b1e79-8930-4ef0-817f-5c8305d69b77 admin admin] PATH_INFO: `/v3/users/4798b64da9a54711a2edf6679fd4f32e` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:31:54.332816 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c63b1e79-8930-4ef0-817f-5c8305d69b77 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4798b64da9a54711a2edf6679fd4f32e)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:31:54.334759 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c63b1e79-8930-4ef0-817f-5c8305d69b77 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:31:54.374569 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ea4a1d-3e16-49fc-b0ee-9c54988ebdaf admin admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:31:54.380404 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543b4ceb-7cef-41c5-a615-96f1665e67e2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20d1327553404b80aecee87c78fcb52b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '559b3e713f5844b6ae37151d56dd419f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:31:54.381172 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543b4ceb-7cef-41c5-a615-96f1665e67e2 service cinder] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:31:54.383308 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543b4ceb-7cef-41c5-a615-96f1665e67e2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:31:54.383681 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543b4ceb-7cef-41c5-a615-96f1665e67e2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:31:54.427028 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-c63b1e79-8930-4ef0-817f-5c8305d69b77 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4798b64da9a54711a2edf6679fd4f32e'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:31:54.448727 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-c63b1e79-8930-4ef0-817f-5c8305d69b77 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4798b64da9a54711a2edf6679fd4f32e'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:31:54.450830 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-c63b1e79-8930-4ef0-817f-5c8305d69b77 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4798b64da9a54711a2edf6679fd4f32e'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:31:54.452721 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-c63b1e79-8930-4ef0-817f-5c8305d69b77 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4798b64da9a54711a2edf6679fd4f32e'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:31:54.482441 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-c63b1e79-8930-4ef0-817f-5c8305d69b77 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4798b64da9a54711a2edf6679fd4f32e'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:31:54.509965 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 932/1872] 10.222.0.49 () {58 vars in 1223 bytes} [Wed Jun 15 10:31:54 2022] DELETE /identity/v3/users/4798b64da9a54711a2edf6679fd4f32e => generated 0 bytes in 313 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:31:54.509965 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3790af2-278b-4ae9-8659-b98cecc243ef None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:31:54.509965 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3790af2-278b-4ae9-8659-b98cecc243ef None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:31:54.509965 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3790af2-278b-4ae9-8659-b98cecc243ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:31:54.536216 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:31:54.541708 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-543b4ceb-7cef-41c5-a615-96f1665e67e2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:31:54.543221 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-543b4ceb-7cef-41c5-a615-96f1665e67e2 service cinder] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:31:54.547912 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-f3790af2-278b-4ae9-8659-b98cecc243ef None None] MFA Rules not processed for user `c762e4c31586473eb81ab8d9298600f6`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:31:54.549165 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-f3790af2-278b-4ae9-8659-b98cecc243ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:31:54.568271 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 941/1873] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Jun 15 10:31:54 2022] GET /identity/v3/auth/tokens => generated 3303 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:31:54.569604 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18dbe6e2-8526-46ce-81e1-c08044374a3a None None] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:31:54.569892 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18dbe6e2-8526-46ce-81e1-c08044374a3a None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:31:54.570267 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18dbe6e2-8526-46ce-81e1-c08044374a3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:31:54.589985 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.auth.core [None req-18dbe6e2-8526-46ce-81e1-c08044374a3a None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:31:54.591319 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-18dbe6e2-8526-46ce-81e1-c08044374a3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:31:54.695159 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 933/1874] 10.222.0.49 () {58 vars in 957 bytes} [Wed Jun 15 10:31:54 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:31:54.695159 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3790af2-278b-4ae9-8659-b98cecc243ef None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:31:54.698018 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-f3790af2-278b-4ae9-8659-b98cecc243ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:31:54.714638 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 942/1875] 10.222.0.49 () {58 vars in 957 bytes} [Wed Jun 15 10:31:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:31:54.717559 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18dbe6e2-8526-46ce-81e1-c08044374a3a None None] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:31:54.721141 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-18dbe6e2-8526-46ce-81e1-c08044374a3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:31:54.749403 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18dbe6e2-8526-46ce-81e1-c08044374a3a None None] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:31:54.756101 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de4fd28-a753-40da-801c-90377fdb8bed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:31:54.757253 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de4fd28-a753-40da-801c-90377fdb8bed admin admin] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:31:54.757253 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de4fd28-a753-40da-801c-90377fdb8bed admin admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:31:54.757447 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de4fd28-a753-40da-801c-90377fdb8bed admin admin] PATH_INFO: `/v3/projects` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:31:54.758409 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de4fd28-a753-40da-801c-90377fdb8bed admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:31:54.762040 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de4fd28-a753-40da-801c-90377fdb8bed admin admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:31:54.804298 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 943/1876] 10.222.0.49 () {60 vars in 1152 bytes} [Wed Jun 15 10:31:54 2022] POST /identity/v3/projects => generated 385 bytes in 86 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 10:31:54.804298 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de4fd28-a753-40da-801c-90377fdb8bed admin admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:31:54.807782 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-4de4fd28-a753-40da-801c-90377fdb8bed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:31:54.815752 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:31:54.843868 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3790af2-278b-4ae9-8659-b98cecc243ef None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:31:54.896134 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de4fd28-a753-40da-801c-90377fdb8bed admin admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:31:54.900449 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb533189-b32c-4508-b1e0-b69d3799d3cf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0cb44a17d205431793e9a8791e477a32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '559b3e713f5844b6ae37151d56dd419f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:31:54.901461 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb533189-b32c-4508-b1e0-b69d3799d3cf service neutron] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:31:54.901563 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb533189-b32c-4508-b1e0-b69d3799d3cf service neutron] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:31:54.901906 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb533189-b32c-4508-b1e0-b69d3799d3cf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:31:54.972795 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7618ff4-abc1-46c6-959c-de9ac9bcc02e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20d1327553404b80aecee87c78fcb52b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '559b3e713f5844b6ae37151d56dd419f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:31:54.974404 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7618ff4-abc1-46c6-959c-de9ac9bcc02e service cinder] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:31:54.974749 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7618ff4-abc1-46c6-959c-de9ac9bcc02e service cinder] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:31:54.975029 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7618ff4-abc1-46c6-959c-de9ac9bcc02e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:31:54.978947 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-a7618ff4-abc1-46c6-959c-de9ac9bcc02e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:31:54.989248 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb533189-b32c-4508-b1e0-b69d3799d3cf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:31:54.991524 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb533189-b32c-4508-b1e0-b69d3799d3cf service neutron] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:31:55.004962 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 944/1877] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Jun 15 10:31:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 203 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:31:55.006307 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb533189-b32c-4508-b1e0-b69d3799d3cf service neutron] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:31:55.018174 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb533189-b32c-4508-b1e0-b69d3799d3cf service neutron] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:31:55.024842 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66eb014e-56e6-45c2-ac59-f8c0538f602c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:31:55.025822 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66eb014e-56e6-45c2-ac59-f8c0538f602c admin admin] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:31:55.026293 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66eb014e-56e6-45c2-ac59-f8c0538f602c admin admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:31:55.026740 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66eb014e-56e6-45c2-ac59-f8c0538f602c admin admin] PATH_INFO: `/v3/users` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:31:55.028086 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66eb014e-56e6-45c2-ac59-f8c0538f602c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:31:55.030612 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66eb014e-56e6-45c2-ac59-f8c0538f602c admin admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:31:55.059238 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 945/1878] 10.222.0.49 () {60 vars in 1143 bytes} [Wed Jun 15 10:31:55 2022] POST /identity/v3/users => generated 299 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 10:31:55.060629 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66eb014e-56e6-45c2-ac59-f8c0538f602c admin admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:31:55.063160 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-66eb014e-56e6-45c2-ac59-f8c0538f602c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:31:55.081496 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66eb014e-56e6-45c2-ac59-f8c0538f602c admin admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:31:55.093387 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2069e6ff-4b59-4bae-adcf-b9a90c28b4a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:31:55.095465 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2069e6ff-4b59-4bae-adcf-b9a90c28b4a9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:31:55.095935 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2069e6ff-4b59-4bae-adcf-b9a90c28b4a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:31:55.097124 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2069e6ff-4b59-4bae-adcf-b9a90c28b4a9 admin admin] PATH_INFO: `/v3/projects/38a3de944c244ae4a8b80f68bda69515` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:31:55.099018 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2069e6ff-4b59-4bae-adcf-b9a90c28b4a9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=38a3de944c244ae4a8b80f68bda69515)` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:31:55.102807 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2069e6ff-4b59-4bae-adcf-b9a90c28b4a9 admin admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:31:55.108187 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7618ff4-abc1-46c6-959c-de9ac9bcc02e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:31:55.109846 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7618ff4-abc1-46c6-959c-de9ac9bcc02e service cinder] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:31:55.125223 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 934/1879] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Jun 15 10:31:54 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 432 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:31:55.127516 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7618ff4-abc1-46c6-959c-de9ac9bcc02e service cinder] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:31:55.155270 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7618ff4-abc1-46c6-959c-de9ac9bcc02e service cinder] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:31:55.167725 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587d9e7e-404f-45b7-a7f1-fa1b60c03a7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:31:55.169570 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587d9e7e-404f-45b7-a7f1-fa1b60c03a7c admin admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:31:55.169862 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587d9e7e-404f-45b7-a7f1-fa1b60c03a7c admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:31:55.170137 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587d9e7e-404f-45b7-a7f1-fa1b60c03a7c admin admin] PATH_INFO: `/v3/roles` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:31:55.171171 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587d9e7e-404f-45b7-a7f1-fa1b60c03a7c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:31:55.172753 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587d9e7e-404f-45b7-a7f1-fa1b60c03a7c admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:31:55.182445 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-2069e6ff-4b59-4bae-adcf-b9a90c28b4a9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '38a3de944c244ae4a8b80f68bda69515', 'request_id': 'req-2069e6ff-4b59-4bae-adcf-b9a90c28b4a9'} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:31:55.192593 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-2069e6ff-4b59-4bae-adcf-b9a90c28b4a9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '38a3de944c244ae4a8b80f68bda69515', 'request_id': 'req-2069e6ff-4b59-4bae-adcf-b9a90c28b4a9'} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:31:55.195686 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:31:55.206801 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 935/1880] 10.222.0.49 () {58 vars in 1121 bytes} [Wed Jun 15 10:31:55 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 10:31:55.207174 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587d9e7e-404f-45b7-a7f1-fa1b60c03a7c admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:31:55.220427 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587d9e7e-404f-45b7-a7f1-fa1b60c03a7c admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:31:55.225166 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-2069e6ff-4b59-4bae-adcf-b9a90c28b4a9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '38a3de944c244ae4a8b80f68bda69515', 'request_id': 'req-2069e6ff-4b59-4bae-adcf-b9a90c28b4a9'} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:31:55.243282 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-2069e6ff-4b59-4bae-adcf-b9a90c28b4a9 admin admin] The token cache is being invalidate because project 38a3de944c244ae4a8b80f68bda69515 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80119) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 10:31:55.243577 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-2069e6ff-4b59-4bae-adcf-b9a90c28b4a9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:31:55.244382 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-2069e6ff-4b59-4bae-adcf-b9a90c28b4a9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:31:55.252122 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 946/1881] 10.222.0.49 () {58 vars in 1232 bytes} [Wed Jun 15 10:31:55 2022] DELETE /identity/v3/projects/38a3de944c244ae4a8b80f68bda69515 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:31:55.257251 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2069e6ff-4b59-4bae-adcf-b9a90c28b4a9 admin admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:31:55.260819 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-2069e6ff-4b59-4bae-adcf-b9a90c28b4a9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:31:55.329742 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:31:55.401378 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fcab168-e419-49ca-a113-d78f270546f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:31:55.404770 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fcab168-e419-49ca-a113-d78f270546f5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:31:55.405259 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fcab168-e419-49ca-a113-d78f270546f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:31:55.406397 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fcab168-e419-49ca-a113-d78f270546f5 admin admin] PATH_INFO: `/v3/projects/329bb1184c134d1d904340b8a8c053cc/users/4b360a68611f45238d04d802f0595d01/roles/8051174e7a7947d5b991ebd793c2e543` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:31:55.482723 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fcab168-e419-49ca-a113-d78f270546f5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=329bb1184c134d1d904340b8a8c053cc, user_id=4b360a68611f45238d04d802f0595d01, role_id=8051174e7a7947d5b991ebd793c2e543)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:31:55.482947 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2069e6ff-4b59-4bae-adcf-b9a90c28b4a9 admin admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:31:55.493730 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fcab168-e419-49ca-a113-d78f270546f5 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:31:55.505234 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0bf51cc-eb56-4ca5-9a9f-1ab4a0b37dfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:31:55.507675 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bf51cc-eb56-4ca5-9a9f-1ab4a0b37dfe admin admin] REQUEST_METHOD: `DELETE` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:31:55.509114 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bf51cc-eb56-4ca5-9a9f-1ab4a0b37dfe admin admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:31:55.510966 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bf51cc-eb56-4ca5-9a9f-1ab4a0b37dfe admin admin] PATH_INFO: `/v3/users/128adcb269f544a5bb7c2f5a232c6b08` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:31:55.513381 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0bf51cc-eb56-4ca5-9a9f-1ab4a0b37dfe admin admin] RBAC: Authorizing `identity:delete_user(user_id=128adcb269f544a5bb7c2f5a232c6b08)` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:31:55.520731 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0bf51cc-eb56-4ca5-9a9f-1ab4a0b37dfe admin admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:31:55.529120 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 936/1882] 10.222.0.49 () {60 vars in 1482 bytes} [Wed Jun 15 10:31:55 2022] PUT /identity/v3/projects/329bb1184c134d1d904340b8a8c053cc/users/4b360a68611f45238d04d802f0595d01/roles/8051174e7a7947d5b991ebd793c2e543 => generated 0 bytes in 319 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:31:55.531439 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fcab168-e419-49ca-a113-d78f270546f5 admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:31:55.534537 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-3fcab168-e419-49ca-a113-d78f270546f5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:31:55.652527 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-d0bf51cc-eb56-4ca5-9a9f-1ab4a0b37dfe admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '128adcb269f544a5bb7c2f5a232c6b08'} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:31:55.682215 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-d0bf51cc-eb56-4ca5-9a9f-1ab4a0b37dfe admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '128adcb269f544a5bb7c2f5a232c6b08'} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:31:55.684070 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-d0bf51cc-eb56-4ca5-9a9f-1ab4a0b37dfe admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '128adcb269f544a5bb7c2f5a232c6b08'} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:31:55.684686 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-d0bf51cc-eb56-4ca5-9a9f-1ab4a0b37dfe admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '128adcb269f544a5bb7c2f5a232c6b08'} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:31:55.708206 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fcab168-e419-49ca-a113-d78f270546f5 admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:31:55.710251 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-d0bf51cc-eb56-4ca5-9a9f-1ab4a0b37dfe admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '128adcb269f544a5bb7c2f5a232c6b08'} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:31:55.722506 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22a2dd8-54a1-47ae-96fe-8374329d868e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:31:55.723284 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22a2dd8-54a1-47ae-96fe-8374329d868e admin admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:31:55.723569 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22a2dd8-54a1-47ae-96fe-8374329d868e admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:31:55.723845 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22a2dd8-54a1-47ae-96fe-8374329d868e admin admin] PATH_INFO: `/v3/roles` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:31:55.724597 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b22a2dd8-54a1-47ae-96fe-8374329d868e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:31:55.729192 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b22a2dd8-54a1-47ae-96fe-8374329d868e admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:31:55.763685 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 937/1883] 10.222.0.49 () {58 vars in 1121 bytes} [Wed Jun 15 10:31:55 2022] GET /identity/v3/roles => generated 1420 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 10:31:55.764235 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22a2dd8-54a1-47ae-96fe-8374329d868e admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:31:55.771177 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-b22a2dd8-54a1-47ae-96fe-8374329d868e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:31:55.787161 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 947/1884] 10.222.0.49 () {58 vars in 1223 bytes} [Wed Jun 15 10:31:55 2022] DELETE /identity/v3/users/128adcb269f544a5bb7c2f5a232c6b08 => generated 0 bytes in 527 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:31:55.809015 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22a2dd8-54a1-47ae-96fe-8374329d868e admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:31:55.981013 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9288388-4c76-4887-950e-dba3ebd7a51c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:31:55.982133 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9288388-4c76-4887-950e-dba3ebd7a51c admin admin] REQUEST_METHOD: `PUT` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:31:55.982802 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9288388-4c76-4887-950e-dba3ebd7a51c admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:31:55.983103 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9288388-4c76-4887-950e-dba3ebd7a51c admin admin] PATH_INFO: `/v3/projects/329bb1184c134d1d904340b8a8c053cc/users/4b360a68611f45238d04d802f0595d01/roles/8051174e7a7947d5b991ebd793c2e543` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:31:55.986630 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9288388-4c76-4887-950e-dba3ebd7a51c admin admin] RBAC: Authorizing `identity:create_grant(project_id=329bb1184c134d1d904340b8a8c053cc, user_id=4b360a68611f45238d04d802f0595d01, role_id=8051174e7a7947d5b991ebd793c2e543)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:31:55.992236 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9288388-4c76-4887-950e-dba3ebd7a51c admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:31:56.022379 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 938/1885] 10.222.0.49 () {60 vars in 1482 bytes} [Wed Jun 15 10:31:55 2022] PUT /identity/v3/projects/329bb1184c134d1d904340b8a8c053cc/users/4b360a68611f45238d04d802f0595d01/roles/8051174e7a7947d5b991ebd793c2e543 => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:31:56.043489 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f81a6a7-5406-4abc-b22c-c1782fd0b9cb None None] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:31:56.043489 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f81a6a7-5406-4abc-b22c-c1782fd0b9cb None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:31:56.043489 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f81a6a7-5406-4abc-b22c-c1782fd0b9cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:31:56.089588 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.auth.core [None req-1f81a6a7-5406-4abc-b22c-c1782fd0b9cb None None] MFA Rules not processed for user `4b360a68611f45238d04d802f0595d01`. Rule list: `[]` (Enabled: `True`). {{(pid=80119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:31:56.093055 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-1f81a6a7-5406-4abc-b22c-c1782fd0b9cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:31:56.299044 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 948/1886] 10.222.0.49 () {58 vars in 957 bytes} [Wed Jun 15 10:31:56 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 261 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:31:56.312184 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9288388-4c76-4887-950e-dba3ebd7a51c admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:31:56.314195 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-e9288388-4c76-4887-950e-dba3ebd7a51c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:31:56.348830 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:31:56.361637 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ee1e3a-edb2-421e-97fb-3b2ced5420cd None None] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:31:56.362127 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ee1e3a-edb2-421e-97fb-3b2ced5420cd None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:31:56.365718 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ee1e3a-edb2-421e-97fb-3b2ced5420cd None None] PATH_INFO: `/` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:31:56.370833 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 949/1887] 10.222.0.49 () {54 vars in 921 bytes} [Wed Jun 15 10:31:56 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:31:56.391299 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f699223-4d60-48c4-ab47-b50e3d2de7cb None None] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:31:56.392231 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f699223-4d60-48c4-ab47-b50e3d2de7cb None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:31:56.395691 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f699223-4d60-48c4-ab47-b50e3d2de7cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:31:56.435275 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.auth.core [None req-9f699223-4d60-48c4-ab47-b50e3d2de7cb None None] MFA Rules not processed for user `5ac54528370d4747b6e8777fba9d0c82`. Rule list: `[]` (Enabled: `True`). {{(pid=80119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:31:56.436685 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-9f699223-4d60-48c4-ab47-b50e3d2de7cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:31:56.452775 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9288388-4c76-4887-950e-dba3ebd7a51c admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:31:56.458969 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f041f97-508b-4a28-bbfb-b8ebd6e2357b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20d1327553404b80aecee87c78fcb52b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '559b3e713f5844b6ae37151d56dd419f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:31:56.459727 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f041f97-508b-4a28-bbfb-b8ebd6e2357b service cinder] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:31:56.460016 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f041f97-508b-4a28-bbfb-b8ebd6e2357b service cinder] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:31:56.460300 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f041f97-508b-4a28-bbfb-b8ebd6e2357b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:31:56.493322 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f041f97-508b-4a28-bbfb-b8ebd6e2357b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:31:56.495136 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f041f97-508b-4a28-bbfb-b8ebd6e2357b service cinder] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:31:56.497651 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:31:56.531307 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 939/1888] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Jun 15 10:31:56 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 218 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:31:56.582192 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f041f97-508b-4a28-bbfb-b8ebd6e2357b service cinder] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:31:56.619951 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 950/1889] 10.222.0.49 () {58 vars in 1018 bytes} [Wed Jun 15 10:31:56 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:31:56.679570 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f041f97-508b-4a28-bbfb-b8ebd6e2357b service cinder] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:31:56.700623 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feeff02b-e218-486f-bf81-67336534d3eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:31:56.703008 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feeff02b-e218-486f-bf81-67336534d3eb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:31:56.705300 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feeff02b-e218-486f-bf81-67336534d3eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:31:56.705935 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feeff02b-e218-486f-bf81-67336534d3eb admin admin] PATH_INFO: `/v3/users/c762e4c31586473eb81ab8d9298600f6` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:31:56.710914 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feeff02b-e218-486f-bf81-67336534d3eb admin admin] RBAC: Authorizing `identity:delete_user(user_id=c762e4c31586473eb81ab8d9298600f6)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:31:56.713850 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feeff02b-e218-486f-bf81-67336534d3eb admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:31:56.786267 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f699223-4d60-48c4-ab47-b50e3d2de7cb None None] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:31:56.787930 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-9f699223-4d60-48c4-ab47-b50e3d2de7cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:31:56.829529 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f699223-4d60-48c4-ab47-b50e3d2de7cb None None] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:31:56.853007 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86516010-204a-4b72-8cdb-284a1ac556b8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ac54528370d4747b6e8777fba9d0c82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '559b3e713f5844b6ae37151d56dd419f', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:31:56.854936 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-feeff02b-e218-486f-bf81-67336534d3eb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c762e4c31586473eb81ab8d9298600f6'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:31:56.855935 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86516010-204a-4b72-8cdb-284a1ac556b8 service nova] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:31:56.856260 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86516010-204a-4b72-8cdb-284a1ac556b8 service nova] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:31:56.856592 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86516010-204a-4b72-8cdb-284a1ac556b8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:31:56.879364 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-feeff02b-e218-486f-bf81-67336534d3eb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c762e4c31586473eb81ab8d9298600f6'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:31:56.881710 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-feeff02b-e218-486f-bf81-67336534d3eb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c762e4c31586473eb81ab8d9298600f6'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:31:56.885277 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-feeff02b-e218-486f-bf81-67336534d3eb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c762e4c31586473eb81ab8d9298600f6'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:31:56.895630 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86516010-204a-4b72-8cdb-284a1ac556b8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:31:56.898687 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86516010-204a-4b72-8cdb-284a1ac556b8 service nova] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:31:56.910173 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-feeff02b-e218-486f-bf81-67336534d3eb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c762e4c31586473eb81ab8d9298600f6'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:31:56.929151 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 951/1890] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Jun 15 10:31:56 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:31:56.951005 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 940/1891] 10.222.0.49 () {58 vars in 1223 bytes} [Wed Jun 15 10:31:56 2022] DELETE /identity/v3/users/c762e4c31586473eb81ab8d9298600f6 => generated 0 bytes in 367 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 10:31:56.953107 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ca10e8-7788-4c68-a02e-2016d340c216 None None] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:31:56.953612 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ca10e8-7788-4c68-a02e-2016d340c216 None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:31:56.954088 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ca10e8-7788-4c68-a02e-2016d340c216 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:31:56.979045 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.auth.core [None req-a1ca10e8-7788-4c68-a02e-2016d340c216 None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:31:56.980361 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-a1ca10e8-7788-4c68-a02e-2016d340c216 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:31:57.035637 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feeff02b-e218-486f-bf81-67336534d3eb admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:31:57.037801 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-feeff02b-e218-486f-bf81-67336534d3eb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:31:57.117147 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 952/1892] 10.222.0.49 () {58 vars in 957 bytes} [Wed Jun 15 10:31:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:31:57.137925 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ca10e8-7788-4c68-a02e-2016d340c216 None None] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:31:57.139982 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-a1ca10e8-7788-4c68-a02e-2016d340c216 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:31:57.160911 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:31:57.230632 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feeff02b-e218-486f-bf81-67336534d3eb admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:31:57.258248 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c408ebfc-9d77-419f-84b2-f6071ba33943 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:31:57.261022 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c408ebfc-9d77-419f-84b2-f6071ba33943 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:31:57.261261 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c408ebfc-9d77-419f-84b2-f6071ba33943 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:31:57.261526 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c408ebfc-9d77-419f-84b2-f6071ba33943 admin admin] PATH_INFO: `/v3/projects/c4d17fc4571545cebec33d03b677a6c1` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:31:57.268085 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c408ebfc-9d77-419f-84b2-f6071ba33943 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c4d17fc4571545cebec33d03b677a6c1)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:31:57.279951 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c408ebfc-9d77-419f-84b2-f6071ba33943 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:31:57.309119 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ca10e8-7788-4c68-a02e-2016d340c216 None None] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:31:57.321964 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3cbca41-904d-48f4-bdd8-a7cf4da4738e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0cb44a17d205431793e9a8791e477a32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '559b3e713f5844b6ae37151d56dd419f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:31:57.322765 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cbca41-904d-48f4-bdd8-a7cf4da4738e service neutron] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:31:57.323372 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cbca41-904d-48f4-bdd8-a7cf4da4738e service neutron] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:31:57.323372 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cbca41-904d-48f4-bdd8-a7cf4da4738e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:31:57.350796 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3cbca41-904d-48f4-bdd8-a7cf4da4738e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:31:57.352854 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3cbca41-904d-48f4-bdd8-a7cf4da4738e service neutron] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:31:57.382049 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 953/1893] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Jun 15 10:31:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 244 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 10:31:57.382696 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-c408ebfc-9d77-419f-84b2-f6071ba33943 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c4d17fc4571545cebec33d03b677a6c1', 'request_id': 'req-c408ebfc-9d77-419f-84b2-f6071ba33943'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:31:57.394202 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-c408ebfc-9d77-419f-84b2-f6071ba33943 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c4d17fc4571545cebec33d03b677a6c1', 'request_id': 'req-c408ebfc-9d77-419f-84b2-f6071ba33943'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:31:57.404017 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-c408ebfc-9d77-419f-84b2-f6071ba33943 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c4d17fc4571545cebec33d03b677a6c1', 'request_id': 'req-c408ebfc-9d77-419f-84b2-f6071ba33943'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:31:57.412654 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3cbca41-904d-48f4-bdd8-a7cf4da4738e service neutron] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:31:57.414762 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-d3cbca41-904d-48f4-bdd8-a7cf4da4738e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:31:57.426039 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-c408ebfc-9d77-419f-84b2-f6071ba33943 admin admin] The token cache is being invalidate because project c4d17fc4571545cebec33d03b677a6c1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80120) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 10:31:57.426340 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-c408ebfc-9d77-419f-84b2-f6071ba33943 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:31:57.427366 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-c408ebfc-9d77-419f-84b2-f6071ba33943 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:31:57.429407 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 941/1894] 10.222.0.49 () {58 vars in 1232 bytes} [Wed Jun 15 10:31:57 2022] DELETE /identity/v3/projects/c4d17fc4571545cebec33d03b677a6c1 => generated 0 bytes in 395 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:31:57.433262 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c408ebfc-9d77-419f-84b2-f6071ba33943 admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:31:57.438406 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-c408ebfc-9d77-419f-84b2-f6071ba33943 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:31:57.439920 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3cbca41-904d-48f4-bdd8-a7cf4da4738e service neutron] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:31:57.557806 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=100 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:31:57.593240 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da4ad18-60c4-4f0c-b2ab-a5f58ed8f0e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:31:57.594900 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da4ad18-60c4-4f0c-b2ab-a5f58ed8f0e2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:31:57.596702 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da4ad18-60c4-4f0c-b2ab-a5f58ed8f0e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:31:57.598018 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da4ad18-60c4-4f0c-b2ab-a5f58ed8f0e2 admin admin] PATH_INFO: `/v3/projects/38ea0331c1d847c09dc1078372d6cba6` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:31:57.603264 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da4ad18-60c4-4f0c-b2ab-a5f58ed8f0e2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=38ea0331c1d847c09dc1078372d6cba6)` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:31:57.609109 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da4ad18-60c4-4f0c-b2ab-a5f58ed8f0e2 admin admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:31:57.621690 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c408ebfc-9d77-419f-84b2-f6071ba33943 admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:31:57.626794 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d74700-105b-4111-9aef-22f65592e04e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:31:57.630958 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d74700-105b-4111-9aef-22f65592e04e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:31:57.632102 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d74700-105b-4111-9aef-22f65592e04e admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:31:57.632390 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d74700-105b-4111-9aef-22f65592e04e admin admin] PATH_INFO: `/v3/users/281bbce0f25a40cf959f30d25c61945f` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:31:57.634148 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d74700-105b-4111-9aef-22f65592e04e admin admin] RBAC: Authorizing `identity:delete_user(user_id=281bbce0f25a40cf959f30d25c61945f)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:31:57.636678 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d74700-105b-4111-9aef-22f65592e04e admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:31:57.697875 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-2da4ad18-60c4-4f0c-b2ab-a5f58ed8f0e2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '38ea0331c1d847c09dc1078372d6cba6', 'request_id': 'req-2da4ad18-60c4-4f0c-b2ab-a5f58ed8f0e2'} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:31:57.714111 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-77d74700-105b-4111-9aef-22f65592e04e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '281bbce0f25a40cf959f30d25c61945f'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:31:57.716061 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-2da4ad18-60c4-4f0c-b2ab-a5f58ed8f0e2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '38ea0331c1d847c09dc1078372d6cba6', 'request_id': 'req-2da4ad18-60c4-4f0c-b2ab-a5f58ed8f0e2'} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:31:57.727266 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-77d74700-105b-4111-9aef-22f65592e04e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '281bbce0f25a40cf959f30d25c61945f'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:31:57.727474 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-2da4ad18-60c4-4f0c-b2ab-a5f58ed8f0e2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '38ea0331c1d847c09dc1078372d6cba6', 'request_id': 'req-2da4ad18-60c4-4f0c-b2ab-a5f58ed8f0e2'} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:31:57.728234 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-77d74700-105b-4111-9aef-22f65592e04e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '281bbce0f25a40cf959f30d25c61945f'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:31:57.729001 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-77d74700-105b-4111-9aef-22f65592e04e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '281bbce0f25a40cf959f30d25c61945f'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:31:57.738786 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-77d74700-105b-4111-9aef-22f65592e04e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '281bbce0f25a40cf959f30d25c61945f'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:31:57.745955 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-2da4ad18-60c4-4f0c-b2ab-a5f58ed8f0e2 admin admin] The token cache is being invalidate because project 38ea0331c1d847c09dc1078372d6cba6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80119) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 10:31:57.746259 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-2da4ad18-60c4-4f0c-b2ab-a5f58ed8f0e2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:31:57.747210 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-2da4ad18-60c4-4f0c-b2ab-a5f58ed8f0e2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:31:57.747897 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:31:57.748990 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 954/1895] 10.222.0.49 () {58 vars in 1232 bytes} [Wed Jun 15 10:31:57 2022] DELETE /identity/v3/projects/38ea0331c1d847c09dc1078372d6cba6 => generated 0 bytes in 337 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:31:57.754098 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c165f84-bc4a-4a61-ab64-c38d460c9fc3 None None] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:31:57.754387 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c165f84-bc4a-4a61-ab64-c38d460c9fc3 None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:31:57.754696 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c165f84-bc4a-4a61-ab64-c38d460c9fc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:31:57.785174 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 942/1896] 10.222.0.49 () {58 vars in 1223 bytes} [Wed Jun 15 10:31:57 2022] DELETE /identity/v3/users/281bbce0f25a40cf959f30d25c61945f => generated 0 bytes in 352 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:31:57.801886 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.auth.core [None req-4c165f84-bc4a-4a61-ab64-c38d460c9fc3 None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:31:57.803599 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-4c165f84-bc4a-4a61-ab64-c38d460c9fc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:31:57.873700 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d74700-105b-4111-9aef-22f65592e04e admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:31:57.875318 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-77d74700-105b-4111-9aef-22f65592e04e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:31:57.915285 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 955/1897] 10.222.0.49 () {58 vars in 957 bytes} [Wed Jun 15 10:31:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 10:31:57.922750 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb22e907-14c8-4b56-b3bd-2b6951409fcd None None] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:31:57.923518 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb22e907-14c8-4b56-b3bd-2b6951409fcd None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:31:57.923518 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb22e907-14c8-4b56-b3bd-2b6951409fcd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:31:57.943030 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.auth.core [None req-fb22e907-14c8-4b56-b3bd-2b6951409fcd None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:31:57.944285 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-fb22e907-14c8-4b56-b3bd-2b6951409fcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:31:57.953597 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 956/1898] 10.222.0.49 () {58 vars in 957 bytes} [Wed Jun 15 10:31:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 10:31:57.957999 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb22e907-14c8-4b56-b3bd-2b6951409fcd None None] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:31:57.973265 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb22e907-14c8-4b56-b3bd-2b6951409fcd None None] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:31:57.983121 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d74700-105b-4111-9aef-22f65592e04e admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:31:57.983887 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e89edf8-3685-498e-a8c3-9654a82ee198 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:31:57.984954 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e89edf8-3685-498e-a8c3-9654a82ee198 admin admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:31:57.985475 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e89edf8-3685-498e-a8c3-9654a82ee198 admin admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:31:57.986059 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e89edf8-3685-498e-a8c3-9654a82ee198 admin admin] PATH_INFO: `/v3/domains` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:31:57.986977 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e89edf8-3685-498e-a8c3-9654a82ee198 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:31:57.987559 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e89edf8-3685-498e-a8c3-9654a82ee198 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:31:57.989884 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e89edf8-3685-498e-a8c3-9654a82ee198 admin admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:31:57.990009 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97392a00-9975-47eb-9adc-bb86ebe0795b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:31:57.990989 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97392a00-9975-47eb-9adc-bb86ebe0795b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:31:57.991388 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97392a00-9975-47eb-9adc-bb86ebe0795b admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:31:57.991722 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97392a00-9975-47eb-9adc-bb86ebe0795b admin admin] PATH_INFO: `/v3/projects/f758049b6de94c10a3dafee0083ba61b` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:31:57.993057 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97392a00-9975-47eb-9adc-bb86ebe0795b admin admin] RBAC: Authorizing `identity:delete_project(project_id=f758049b6de94c10a3dafee0083ba61b)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:31:57.995019 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97392a00-9975-47eb-9adc-bb86ebe0795b admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:31:58.003781 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 957/1899] 10.222.0.49 () {58 vars in 1152 bytes} [Wed Jun 15 10:31:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 10:31:58.007819 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e89edf8-3685-498e-a8c3-9654a82ee198 admin admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:31:58.019880 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e89edf8-3685-498e-a8c3-9654a82ee198 admin admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:31:58.026421 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9016a39-52ca-4938-b08e-8f6c77b93871 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:31:58.027439 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9016a39-52ca-4938-b08e-8f6c77b93871 admin admin] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:31:58.027730 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9016a39-52ca-4938-b08e-8f6c77b93871 admin admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:31:58.028091 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9016a39-52ca-4938-b08e-8f6c77b93871 admin admin] PATH_INFO: `/v3/projects` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:31:58.028902 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9016a39-52ca-4938-b08e-8f6c77b93871 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:31:58.031242 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9016a39-52ca-4938-b08e-8f6c77b93871 admin admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:31:58.053405 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-97392a00-9975-47eb-9adc-bb86ebe0795b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f758049b6de94c10a3dafee0083ba61b', 'request_id': 'req-97392a00-9975-47eb-9adc-bb86ebe0795b'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:31:58.060293 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 958/1900] 10.222.0.49 () {60 vars in 1152 bytes} [Wed Jun 15 10:31:58 2022] POST /identity/v3/projects => generated 381 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 10:31:58.064495 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-97392a00-9975-47eb-9adc-bb86ebe0795b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f758049b6de94c10a3dafee0083ba61b', 'request_id': 'req-97392a00-9975-47eb-9adc-bb86ebe0795b'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:31:58.064667 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9016a39-52ca-4938-b08e-8f6c77b93871 admin admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:31:58.072552 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-97392a00-9975-47eb-9adc-bb86ebe0795b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f758049b6de94c10a3dafee0083ba61b', 'request_id': 'req-97392a00-9975-47eb-9adc-bb86ebe0795b'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:31:58.084970 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-97392a00-9975-47eb-9adc-bb86ebe0795b admin admin] The token cache is being invalidate because project f758049b6de94c10a3dafee0083ba61b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80120) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 10:31:58.085348 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-97392a00-9975-47eb-9adc-bb86ebe0795b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:31:58.086201 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-97392a00-9975-47eb-9adc-bb86ebe0795b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:31:58.088019 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 943/1901] 10.222.0.49 () {58 vars in 1232 bytes} [Wed Jun 15 10:31:57 2022] DELETE /identity/v3/projects/f758049b6de94c10a3dafee0083ba61b => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:31:58.092122 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97392a00-9975-47eb-9adc-bb86ebe0795b admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:31:58.094307 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-97392a00-9975-47eb-9adc-bb86ebe0795b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:31:58.102886 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:31:58.144072 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9016a39-52ca-4938-b08e-8f6c77b93871 admin admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:31:58.149529 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1c2d13-6add-4a21-b014-8fbb86f2d146 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:31:58.150220 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1c2d13-6add-4a21-b014-8fbb86f2d146 admin admin] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:31:58.150619 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1c2d13-6add-4a21-b014-8fbb86f2d146 admin admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:31:58.150904 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1c2d13-6add-4a21-b014-8fbb86f2d146 admin admin] PATH_INFO: `/v3/users` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:31:58.151677 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c1c2d13-6add-4a21-b014-8fbb86f2d146 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:31:58.154161 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c1c2d13-6add-4a21-b014-8fbb86f2d146 admin admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:31:58.171964 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97392a00-9975-47eb-9adc-bb86ebe0795b admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:31:58.178345 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26a1bae-8b6c-496f-a2bb-9fd3c6738a34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:31:58.179402 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26a1bae-8b6c-496f-a2bb-9fd3c6738a34 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:31:58.180340 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26a1bae-8b6c-496f-a2bb-9fd3c6738a34 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:31:58.183751 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26a1bae-8b6c-496f-a2bb-9fd3c6738a34 admin admin] PATH_INFO: `/v3/users/4b360a68611f45238d04d802f0595d01` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:31:58.184364 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 959/1902] 10.222.0.49 () {60 vars in 1143 bytes} [Wed Jun 15 10:31:58 2022] POST /identity/v3/users => generated 291 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 10:31:58.184364 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1c2d13-6add-4a21-b014-8fbb86f2d146 admin admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:31:58.184836 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a26a1bae-8b6c-496f-a2bb-9fd3c6738a34 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4b360a68611f45238d04d802f0595d01)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:31:58.185791 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-8c1c2d13-6add-4a21-b014-8fbb86f2d146 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:31:58.186772 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a26a1bae-8b6c-496f-a2bb-9fd3c6738a34 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:31:58.198639 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1c2d13-6add-4a21-b014-8fbb86f2d146 admin admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:31:58.203626 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b56907-903a-4778-9eb8-fa150139daf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:31:58.203778 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:31:58.204519 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b56907-903a-4778-9eb8-fa150139daf8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:31:58.204774 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b56907-903a-4778-9eb8-fa150139daf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:31:58.205052 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b56907-903a-4778-9eb8-fa150139daf8 admin admin] PATH_INFO: `/v3/projects/5b00baf3d6d940bfa8531957dcb0b7e2` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:31:58.206291 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b56907-903a-4778-9eb8-fa150139daf8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5b00baf3d6d940bfa8531957dcb0b7e2)` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:31:58.208268 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b56907-903a-4778-9eb8-fa150139daf8 admin admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:31:58.249226 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-a26a1bae-8b6c-496f-a2bb-9fd3c6738a34 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4b360a68611f45238d04d802f0595d01'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:31:58.256476 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-a26a1bae-8b6c-496f-a2bb-9fd3c6738a34 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4b360a68611f45238d04d802f0595d01'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:31:58.257279 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-a26a1bae-8b6c-496f-a2bb-9fd3c6738a34 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4b360a68611f45238d04d802f0595d01'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:31:58.258588 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-a26a1bae-8b6c-496f-a2bb-9fd3c6738a34 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4b360a68611f45238d04d802f0595d01'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:31:58.259958 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-66b56907-903a-4778-9eb8-fa150139daf8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5b00baf3d6d940bfa8531957dcb0b7e2', 'request_id': 'req-66b56907-903a-4778-9eb8-fa150139daf8'} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:31:58.267617 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-a26a1bae-8b6c-496f-a2bb-9fd3c6738a34 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4b360a68611f45238d04d802f0595d01'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:31:58.270021 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-66b56907-903a-4778-9eb8-fa150139daf8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5b00baf3d6d940bfa8531957dcb0b7e2', 'request_id': 'req-66b56907-903a-4778-9eb8-fa150139daf8'} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:31:58.276656 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-66b56907-903a-4778-9eb8-fa150139daf8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5b00baf3d6d940bfa8531957dcb0b7e2', 'request_id': 'req-66b56907-903a-4778-9eb8-fa150139daf8'} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:31:58.299965 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 944/1903] 10.222.0.49 () {58 vars in 1223 bytes} [Wed Jun 15 10:31:58 2022] DELETE /identity/v3/users/4b360a68611f45238d04d802f0595d01 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:31:58.301097 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26a1bae-8b6c-496f-a2bb-9fd3c6738a34 admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:31:58.302158 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-66b56907-903a-4778-9eb8-fa150139daf8 admin admin] The token cache is being invalidate because project 5b00baf3d6d940bfa8531957dcb0b7e2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80119) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 10:31:58.302461 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-66b56907-903a-4778-9eb8-fa150139daf8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:31:58.303015 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-a26a1bae-8b6c-496f-a2bb-9fd3c6738a34 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:31:58.303575 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-66b56907-903a-4778-9eb8-fa150139daf8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:31:58.311472 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 960/1904] 10.222.0.49 () {58 vars in 1232 bytes} [Wed Jun 15 10:31:58 2022] DELETE /identity/v3/projects/5b00baf3d6d940bfa8531957dcb0b7e2 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:31:58.312922 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d68db0-1695-495c-94c8-9d4529c392e0 None None] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:31:58.313288 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d68db0-1695-495c-94c8-9d4529c392e0 None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:31:58.313635 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d68db0-1695-495c-94c8-9d4529c392e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:31:58.342461 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.auth.core [None req-41d68db0-1695-495c-94c8-9d4529c392e0 None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:31:58.344251 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-41d68db0-1695-495c-94c8-9d4529c392e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:31:58.429674 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26a1bae-8b6c-496f-a2bb-9fd3c6738a34 admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:31:58.435828 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6575c5-2276-48f5-bb31-3718487e3a6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:31:58.436507 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6575c5-2276-48f5-bb31-3718487e3a6c admin admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:31:58.436805 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6575c5-2276-48f5-bb31-3718487e3a6c admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:31:58.437705 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6575c5-2276-48f5-bb31-3718487e3a6c admin admin] PATH_INFO: `/v3/roles` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:31:58.438463 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6575c5-2276-48f5-bb31-3718487e3a6c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:31:58.440283 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6575c5-2276-48f5-bb31-3718487e3a6c admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:31:58.455194 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 945/1905] 10.222.0.49 () {58 vars in 1121 bytes} [Wed Jun 15 10:31:58 2022] GET /identity/v3/roles => generated 1420 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 10:31:58.456124 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6575c5-2276-48f5-bb31-3718487e3a6c admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:31:58.457980 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-7c6575c5-2276-48f5-bb31-3718487e3a6c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:31:58.463992 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 961/1906] 10.222.0.49 () {58 vars in 957 bytes} [Wed Jun 15 10:31:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:31:58.472504 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e603b35-363c-4f0f-adab-54e29f5232af None None] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:31:58.473027 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e603b35-363c-4f0f-adab-54e29f5232af None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:31:58.473315 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6575c5-2276-48f5-bb31-3718487e3a6c admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:31:58.473634 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e603b35-363c-4f0f-adab-54e29f5232af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:31:58.482631 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9603b2e9-c407-4b63-b53b-2274424cb673 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:31:58.483364 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9603b2e9-c407-4b63-b53b-2274424cb673 admin admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:31:58.483658 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9603b2e9-c407-4b63-b53b-2274424cb673 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:31:58.483993 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9603b2e9-c407-4b63-b53b-2274424cb673 admin admin] PATH_INFO: `/v3/roles` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:31:58.484732 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9603b2e9-c407-4b63-b53b-2274424cb673 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:31:58.486440 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9603b2e9-c407-4b63-b53b-2274424cb673 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:31:58.509607 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 946/1907] 10.222.0.49 () {58 vars in 1121 bytes} [Wed Jun 15 10:31:58 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 10:31:58.511620 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9603b2e9-c407-4b63-b53b-2274424cb673 admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:31:58.513559 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.auth.core [None req-4e603b35-363c-4f0f-adab-54e29f5232af None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:31:58.516679 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-4e603b35-363c-4f0f-adab-54e29f5232af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:31:58.526429 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 962/1908] 10.222.0.49 () {58 vars in 957 bytes} [Wed Jun 15 10:31:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 10:31:58.530961 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e603b35-363c-4f0f-adab-54e29f5232af None None] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:31:58.546754 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e603b35-363c-4f0f-adab-54e29f5232af None None] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:31:58.553602 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9603b2e9-c407-4b63-b53b-2274424cb673 admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:31:58.554543 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e4febd-3286-4303-b5e2-3f0eb2c66506 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:31:58.556010 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e4febd-3286-4303-b5e2-3f0eb2c66506 admin admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:31:58.556459 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e4febd-3286-4303-b5e2-3f0eb2c66506 admin admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:31:58.556990 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e4febd-3286-4303-b5e2-3f0eb2c66506 admin admin] PATH_INFO: `/v3/domains` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:31:58.557871 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e4febd-3286-4303-b5e2-3f0eb2c66506 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:31:58.558334 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e4febd-3286-4303-b5e2-3f0eb2c66506 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:31:58.560500 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e4febd-3286-4303-b5e2-3f0eb2c66506 admin admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:31:58.565846 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af958dc-5e19-444c-aab0-4935de487cf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:31:58.567325 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af958dc-5e19-444c-aab0-4935de487cf2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:31:58.569482 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af958dc-5e19-444c-aab0-4935de487cf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:31:58.569875 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af958dc-5e19-444c-aab0-4935de487cf2 admin admin] PATH_INFO: `/v3/projects/672f282e93b3475e95c5e4bc9a8d2ab9/users/8ca46e4a0945427a89b6944d5004cafc/roles/6876e57cc51c448097ab109d35b2a3da` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:31:58.574992 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 963/1909] 10.222.0.49 () {58 vars in 1152 bytes} [Wed Jun 15 10:31:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 10:31:58.579979 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e4febd-3286-4303-b5e2-3f0eb2c66506 admin admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:31:58.591588 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e4febd-3286-4303-b5e2-3f0eb2c66506 admin admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:31:58.597877 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0d834d-1252-49da-8371-29907aa90090 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:31:58.598555 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0d834d-1252-49da-8371-29907aa90090 admin admin] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:31:58.598872 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0d834d-1252-49da-8371-29907aa90090 admin admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:31:58.599531 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0d834d-1252-49da-8371-29907aa90090 admin admin] PATH_INFO: `/v3/projects` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:31:58.600904 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f0d834d-1252-49da-8371-29907aa90090 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:31:58.603952 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f0d834d-1252-49da-8371-29907aa90090 admin admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:31:58.606290 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af958dc-5e19-444c-aab0-4935de487cf2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=672f282e93b3475e95c5e4bc9a8d2ab9, user_id=8ca46e4a0945427a89b6944d5004cafc, role_id=6876e57cc51c448097ab109d35b2a3da)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:31:58.611048 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af958dc-5e19-444c-aab0-4935de487cf2 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:31:58.649973 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 964/1911] 10.222.0.49 () {60 vars in 1152 bytes} [Wed Jun 15 10:31:58 2022] POST /identity/v3/projects => generated 385 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 10:31:58.651260 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0d834d-1252-49da-8371-29907aa90090 admin admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:31:58.659653 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 947/1911] 10.222.0.49 () {60 vars in 1482 bytes} [Wed Jun 15 10:31:58 2022] PUT /identity/v3/projects/672f282e93b3475e95c5e4bc9a8d2ab9/users/8ca46e4a0945427a89b6944d5004cafc/roles/6876e57cc51c448097ab109d35b2a3da => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:31:58.661286 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11685f05-a9ca-43fd-b1d2-f9c913e060c5 None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:31:58.661600 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11685f05-a9ca-43fd-b1d2-f9c913e060c5 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:31:58.667030 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11685f05-a9ca-43fd-b1d2-f9c913e060c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:31:58.675497 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:31:58.724150 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-11685f05-a9ca-43fd-b1d2-f9c913e060c5 None None] MFA Rules not processed for user `8ca46e4a0945427a89b6944d5004cafc`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:31:58.725395 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-11685f05-a9ca-43fd-b1d2-f9c913e060c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:31:58.795641 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0d834d-1252-49da-8371-29907aa90090 admin admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:31:58.797366 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:DELETE=9,keystone:SELECT=89,keystone:INSERT=2 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:31:58.811495 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8799d0b-5dc2-4c7f-9200-04952ac9cab1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:31:58.812228 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8799d0b-5dc2-4c7f-9200-04952ac9cab1 admin admin] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:31:58.812534 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8799d0b-5dc2-4c7f-9200-04952ac9cab1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:31:58.812962 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8799d0b-5dc2-4c7f-9200-04952ac9cab1 admin admin] PATH_INFO: `/v3/users` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:31:58.813798 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8799d0b-5dc2-4c7f-9200-04952ac9cab1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:31:58.816135 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8799d0b-5dc2-4c7f-9200-04952ac9cab1 admin admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:31:58.837725 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 948/1912] 10.222.0.49 () {58 vars in 957 bytes} [Wed Jun 15 10:31:58 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:31:58.841875 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11685f05-a9ca-43fd-b1d2-f9c913e060c5 None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:31:58.847473 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 965/1913] 10.222.0.49 () {60 vars in 1143 bytes} [Wed Jun 15 10:31:58 2022] POST /identity/v3/users => generated 293 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 10:31:58.848565 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8799d0b-5dc2-4c7f-9200-04952ac9cab1 admin admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:31:58.854262 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11685f05-a9ca-43fd-b1d2-f9c913e060c5 None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:31:58.860051 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16eb994e-e180-4c21-b2b8-c50c10b6a092 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:31:58.860893 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16eb994e-e180-4c21-b2b8-c50c10b6a092 admin admin] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:31:58.861586 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16eb994e-e180-4c21-b2b8-c50c10b6a092 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:31:58.861904 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16eb994e-e180-4c21-b2b8-c50c10b6a092 admin admin] PATH_INFO: `/v3/projects` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:31:58.862710 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16eb994e-e180-4c21-b2b8-c50c10b6a092 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:31:58.864953 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16eb994e-e180-4c21-b2b8-c50c10b6a092 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:31:58.871012 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8799d0b-5dc2-4c7f-9200-04952ac9cab1 admin admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:31:58.895322 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa90b0b0-284f-4a17-a629-354fde8c292d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:31:58.896026 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa90b0b0-284f-4a17-a629-354fde8c292d admin admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:31:58.896321 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa90b0b0-284f-4a17-a629-354fde8c292d admin admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:31:58.896605 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa90b0b0-284f-4a17-a629-354fde8c292d admin admin] PATH_INFO: `/v3/roles` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:31:58.899849 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa90b0b0-284f-4a17-a629-354fde8c292d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:31:58.903449 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa90b0b0-284f-4a17-a629-354fde8c292d admin admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:31:58.929985 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 966/1914] 10.222.0.49 () {58 vars in 1121 bytes} [Wed Jun 15 10:31:58 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 15 10:31:58.930669 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa90b0b0-284f-4a17-a629-354fde8c292d admin admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:31:58.935385 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 949/1915] 10.222.0.49 () {60 vars in 1152 bytes} [Wed Jun 15 10:31:58 2022] POST /identity/v3/projects => generated 381 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 10:31:58.936376 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16eb994e-e180-4c21-b2b8-c50c10b6a092 admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:31:58.960234 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa90b0b0-284f-4a17-a629-354fde8c292d admin admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:31:59.074496 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258258fb-d686-44c4-9c5c-ad8f5517479d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:31:59.076183 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258258fb-d686-44c4-9c5c-ad8f5517479d admin admin] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:31:59.076482 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258258fb-d686-44c4-9c5c-ad8f5517479d admin admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:31:59.076829 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258258fb-d686-44c4-9c5c-ad8f5517479d admin admin] PATH_INFO: `/v3/users` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:31:59.077541 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-258258fb-d686-44c4-9c5c-ad8f5517479d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:31:59.080077 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-258258fb-d686-44c4-9c5c-ad8f5517479d admin admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:31:59.122235 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 967/1916] 10.222.0.49 () {60 vars in 1143 bytes} [Wed Jun 15 10:31:58 2022] POST /identity/v3/users => generated 297 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 10:31:59.125681 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258258fb-d686-44c4-9c5c-ad8f5517479d admin admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:31:59.129557 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-258258fb-d686-44c4-9c5c-ad8f5517479d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:31:59.146126 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16eb994e-e180-4c21-b2b8-c50c10b6a092 admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:31:59.150901 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258258fb-d686-44c4-9c5c-ad8f5517479d admin admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:31:59.153778 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5715eb-8081-4b0e-8280-1318b7888495 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:31:59.154474 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5715eb-8081-4b0e-8280-1318b7888495 admin admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:31:59.155080 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5715eb-8081-4b0e-8280-1318b7888495 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:31:59.155080 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5715eb-8081-4b0e-8280-1318b7888495 admin admin] PATH_INFO: `/v3/roles` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:31:59.155832 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a5715eb-8081-4b0e-8280-1318b7888495 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:31:59.158192 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a5715eb-8081-4b0e-8280-1318b7888495 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:31:59.159038 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b38133-7613-483f-ab75-2fcd441cb795 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:31:59.160947 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b38133-7613-483f-ab75-2fcd441cb795 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:31:59.161417 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b38133-7613-483f-ab75-2fcd441cb795 admin admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:31:59.161842 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b38133-7613-483f-ab75-2fcd441cb795 admin admin] PATH_INFO: `/v3/projects/329bb1184c134d1d904340b8a8c053cc` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:31:59.164970 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b38133-7613-483f-ab75-2fcd441cb795 admin admin] RBAC: Authorizing `identity:delete_project(project_id=329bb1184c134d1d904340b8a8c053cc)` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:31:59.167350 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b38133-7613-483f-ab75-2fcd441cb795 admin admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:31:59.179205 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 950/1917] 10.222.0.49 () {58 vars in 1121 bytes} [Wed Jun 15 10:31:58 2022] GET /identity/v3/roles => generated 1420 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 10:31:59.179205 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5715eb-8081-4b0e-8280-1318b7888495 admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:31:59.191330 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5715eb-8081-4b0e-8280-1318b7888495 admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:31:59.198374 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff724a1b-42fc-412f-86df-0982af0bde90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:31:59.199453 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff724a1b-42fc-412f-86df-0982af0bde90 admin admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:31:59.199778 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff724a1b-42fc-412f-86df-0982af0bde90 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:31:59.200063 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff724a1b-42fc-412f-86df-0982af0bde90 admin admin] PATH_INFO: `/v3/roles` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:31:59.200803 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff724a1b-42fc-412f-86df-0982af0bde90 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:31:59.204635 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff724a1b-42fc-412f-86df-0982af0bde90 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:31:59.251786 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 951/1918] 10.222.0.49 () {58 vars in 1121 bytes} [Wed Jun 15 10:31:59 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 10:31:59.252878 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff724a1b-42fc-412f-86df-0982af0bde90 admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:31:59.262265 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-e0b38133-7613-483f-ab75-2fcd441cb795 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '329bb1184c134d1d904340b8a8c053cc', 'request_id': 'req-e0b38133-7613-483f-ab75-2fcd441cb795'} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:31:59.278363 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff724a1b-42fc-412f-86df-0982af0bde90 admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:31:59.286292 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-e0b38133-7613-483f-ab75-2fcd441cb795 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '329bb1184c134d1d904340b8a8c053cc', 'request_id': 'req-e0b38133-7613-483f-ab75-2fcd441cb795'} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:31:59.295855 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-e0b38133-7613-483f-ab75-2fcd441cb795 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '329bb1184c134d1d904340b8a8c053cc', 'request_id': 'req-e0b38133-7613-483f-ab75-2fcd441cb795'} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:31:59.298779 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=86,keystone:INSERT=7,keystone:DELETE=7 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:31:59.317021 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-e0b38133-7613-483f-ab75-2fcd441cb795 admin admin] The token cache is being invalidate because project 329bb1184c134d1d904340b8a8c053cc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80119) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 10:31:59.317472 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-e0b38133-7613-483f-ab75-2fcd441cb795 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:31:59.326931 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-e0b38133-7613-483f-ab75-2fcd441cb795 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:31:59.326931 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 968/1919] 10.222.0.49 () {58 vars in 1232 bytes} [Wed Jun 15 10:31:59 2022] DELETE /identity/v3/projects/329bb1184c134d1d904340b8a8c053cc => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 10:31:59.327785 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b38133-7613-483f-ab75-2fcd441cb795 admin admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:31:59.331447 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-e0b38133-7613-483f-ab75-2fcd441cb795 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:31:59.345318 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18b06a7-cb10-4bca-af7e-1d75b950a33b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:31:59.346283 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18b06a7-cb10-4bca-af7e-1d75b950a33b admin admin] REQUEST_METHOD: `PUT` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:31:59.346572 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18b06a7-cb10-4bca-af7e-1d75b950a33b admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:31:59.346864 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18b06a7-cb10-4bca-af7e-1d75b950a33b admin admin] PATH_INFO: `/v3/projects/ca2186cc0fd842378a145da171b48417/users/885ecc9a6f9a476eb38a8c33f21e0d61/roles/6876e57cc51c448097ab109d35b2a3da` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:31:59.365461 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a18b06a7-cb10-4bca-af7e-1d75b950a33b admin admin] RBAC: Authorizing `identity:create_grant(project_id=ca2186cc0fd842378a145da171b48417, user_id=885ecc9a6f9a476eb38a8c33f21e0d61, role_id=6876e57cc51c448097ab109d35b2a3da)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:31:59.368611 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a18b06a7-cb10-4bca-af7e-1d75b950a33b admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:31:59.386067 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 952/1920] 10.222.0.49 () {60 vars in 1482 bytes} [Wed Jun 15 10:31:59 2022] PUT /identity/v3/projects/ca2186cc0fd842378a145da171b48417/users/885ecc9a6f9a476eb38a8c33f21e0d61/roles/6876e57cc51c448097ab109d35b2a3da => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:31:59.388333 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4e49d6-5f69-433a-87dc-ca36de96e755 None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:31:59.388333 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4e49d6-5f69-433a-87dc-ca36de96e755 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:31:59.388333 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4e49d6-5f69-433a-87dc-ca36de96e755 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:31:59.410277 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-eb4e49d6-5f69-433a-87dc-ca36de96e755 None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:31:59.411558 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-eb4e49d6-5f69-433a-87dc-ca36de96e755 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:31:59.418468 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b38133-7613-483f-ab75-2fcd441cb795 admin admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:31:59.439805 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:31:59.499219 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 953/1921] 10.222.0.49 () {58 vars in 957 bytes} [Wed Jun 15 10:31:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:31:59.500805 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02508be1-3a11-42e3-b9c9-49f42aee8c36 None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:31:59.501164 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02508be1-3a11-42e3-b9c9-49f42aee8c36 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:31:59.501515 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02508be1-3a11-42e3-b9c9-49f42aee8c36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:31:59.515886 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725257d6-b902-4d0a-9c2d-924ebe8fee06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:31:59.516868 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725257d6-b902-4d0a-9c2d-924ebe8fee06 admin admin] REQUEST_METHOD: `PUT` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:31:59.517183 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725257d6-b902-4d0a-9c2d-924ebe8fee06 admin admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:31:59.517512 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725257d6-b902-4d0a-9c2d-924ebe8fee06 admin admin] PATH_INFO: `/v3/projects/1e7d23ac4f2a4aba974e163d63d9d85b/users/916b4b25302a4415b00a111f6a53f606/roles/8051174e7a7947d5b991ebd793c2e543` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:31:59.533634 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-725257d6-b902-4d0a-9c2d-924ebe8fee06 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1e7d23ac4f2a4aba974e163d63d9d85b, user_id=916b4b25302a4415b00a111f6a53f606, role_id=8051174e7a7947d5b991ebd793c2e543)` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:31:59.535238 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-02508be1-3a11-42e3-b9c9-49f42aee8c36 None None] MFA Rules not processed for user `885ecc9a6f9a476eb38a8c33f21e0d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:31:59.536410 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-725257d6-b902-4d0a-9c2d-924ebe8fee06 admin admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:31:59.536514 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-02508be1-3a11-42e3-b9c9-49f42aee8c36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:31:59.555457 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 969/1922] 10.222.0.49 () {60 vars in 1482 bytes} [Wed Jun 15 10:31:59 2022] PUT /identity/v3/projects/1e7d23ac4f2a4aba974e163d63d9d85b/users/916b4b25302a4415b00a111f6a53f606/roles/8051174e7a7947d5b991ebd793c2e543 => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:31:59.555457 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286e00b0-3510-4292-9d5e-1e72342a5866 None None] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:31:59.555457 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286e00b0-3510-4292-9d5e-1e72342a5866 None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:31:59.555457 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286e00b0-3510-4292-9d5e-1e72342a5866 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:31:59.573427 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.auth.core [None req-286e00b0-3510-4292-9d5e-1e72342a5866 None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:31:59.574647 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-286e00b0-3510-4292-9d5e-1e72342a5866 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:31:59.622643 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 954/1923] 10.222.0.49 () {58 vars in 957 bytes} [Wed Jun 15 10:31:59 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:31:59.622643 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02508be1-3a11-42e3-b9c9-49f42aee8c36 None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:31:59.631866 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02508be1-3a11-42e3-b9c9-49f42aee8c36 None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:31:59.654483 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 970/1924] 10.222.0.49 () {58 vars in 957 bytes} [Wed Jun 15 10:31:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:31:59.655365 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286e00b0-3510-4292-9d5e-1e72342a5866 None None] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:31:59.658868 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-286e00b0-3510-4292-9d5e-1e72342a5866 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:31:59.690396 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:31:59.690616 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20583d6e-5d54-49d6-b2e9-c5f27a13b39a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:31:59.691359 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20583d6e-5d54-49d6-b2e9-c5f27a13b39a admin admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:31:59.691688 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20583d6e-5d54-49d6-b2e9-c5f27a13b39a admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:31:59.691995 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20583d6e-5d54-49d6-b2e9-c5f27a13b39a admin admin] PATH_INFO: `/v3/roles` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:31:59.692864 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20583d6e-5d54-49d6-b2e9-c5f27a13b39a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:31:59.694718 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20583d6e-5d54-49d6-b2e9-c5f27a13b39a admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:31:59.706138 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 955/1925] 10.222.0.49 () {58 vars in 1121 bytes} [Wed Jun 15 10:31:59 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 10:31:59.706879 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20583d6e-5d54-49d6-b2e9-c5f27a13b39a admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:31:59.716566 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20583d6e-5d54-49d6-b2e9-c5f27a13b39a admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:31:59.718887 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286e00b0-3510-4292-9d5e-1e72342a5866 None None] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:31:59.721356 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e01b15-1b86-47c9-867e-56c521cac81c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:31:59.722015 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e01b15-1b86-47c9-867e-56c521cac81c admin admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:31:59.722305 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e01b15-1b86-47c9-867e-56c521cac81c admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:31:59.722523 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e56a50-cfac-4a7a-9934-e7a03c4db80b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20d1327553404b80aecee87c78fcb52b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '559b3e713f5844b6ae37151d56dd419f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:31:59.722639 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e01b15-1b86-47c9-867e-56c521cac81c admin admin] PATH_INFO: `/v3/domains` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:31:59.723206 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e56a50-cfac-4a7a-9934-e7a03c4db80b service cinder] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:31:59.723350 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e01b15-1b86-47c9-867e-56c521cac81c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:31:59.723486 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e56a50-cfac-4a7a-9934-e7a03c4db80b service cinder] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:31:59.723686 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e01b15-1b86-47c9-867e-56c521cac81c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:31:59.723792 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e56a50-cfac-4a7a-9934-e7a03c4db80b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:31:59.725313 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e01b15-1b86-47c9-867e-56c521cac81c admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:31:59.737448 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 956/1926] 10.222.0.49 () {58 vars in 1152 bytes} [Wed Jun 15 10:31:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:31:59.738215 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e01b15-1b86-47c9-867e-56c521cac81c admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:31:59.748440 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e01b15-1b86-47c9-867e-56c521cac81c admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:31:59.753179 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d1d008-1ba5-4765-b6a2-56b98a02d9c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:31:59.754041 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d1d008-1ba5-4765-b6a2-56b98a02d9c5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:31:59.754329 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d1d008-1ba5-4765-b6a2-56b98a02d9c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:31:59.754607 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d1d008-1ba5-4765-b6a2-56b98a02d9c5 admin admin] PATH_INFO: `/v3/projects/1e7d23ac4f2a4aba974e163d63d9d85b/users/916b4b25302a4415b00a111f6a53f606/roles/8051174e7a7947d5b991ebd793c2e543` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:31:59.756636 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d1d008-1ba5-4765-b6a2-56b98a02d9c5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1e7d23ac4f2a4aba974e163d63d9d85b, user_id=916b4b25302a4415b00a111f6a53f606, role_id=8051174e7a7947d5b991ebd793c2e543)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:31:59.759858 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d1d008-1ba5-4765-b6a2-56b98a02d9c5 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:31:59.770975 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 957/1927] 10.222.0.49 () {60 vars in 1482 bytes} [Wed Jun 15 10:31:59 2022] PUT /identity/v3/projects/1e7d23ac4f2a4aba974e163d63d9d85b/users/916b4b25302a4415b00a111f6a53f606/roles/8051174e7a7947d5b991ebd793c2e543 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:31:59.770975 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d1d008-1ba5-4765-b6a2-56b98a02d9c5 admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:31:59.800519 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e56a50-cfac-4a7a-9934-e7a03c4db80b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:31:59.802004 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e56a50-cfac-4a7a-9934-e7a03c4db80b service cinder] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:31:59.811227 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:31:59.812122 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 971/1928] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Jun 15 10:31:59 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:31:59.813555 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a13eb2-94e3-4222-b4fa-ea251a58483d None None] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:31:59.813835 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a13eb2-94e3-4222-b4fa-ea251a58483d None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:31:59.814244 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a13eb2-94e3-4222-b4fa-ea251a58483d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:31:59.846442 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.auth.core [None req-60a13eb2-94e3-4222-b4fa-ea251a58483d None None] MFA Rules not processed for user `916b4b25302a4415b00a111f6a53f606`. Rule list: `[]` (Enabled: `True`). {{(pid=80119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:31:59.847931 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-60a13eb2-94e3-4222-b4fa-ea251a58483d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:31:59.850557 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d1d008-1ba5-4765-b6a2-56b98a02d9c5 admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:31:59.858470 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131b6bf4-6c02-49cf-89b8-beed5ae8d38f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:31:59.859180 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131b6bf4-6c02-49cf-89b8-beed5ae8d38f admin admin] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:31:59.859474 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131b6bf4-6c02-49cf-89b8-beed5ae8d38f admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:31:59.859751 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131b6bf4-6c02-49cf-89b8-beed5ae8d38f admin admin] PATH_INFO: `/v3/projects` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:31:59.860456 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131b6bf4-6c02-49cf-89b8-beed5ae8d38f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:31:59.862368 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131b6bf4-6c02-49cf-89b8-beed5ae8d38f admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:31:59.895260 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 958/1929] 10.222.0.49 () {60 vars in 1152 bytes} [Wed Jun 15 10:31:59 2022] POST /identity/v3/projects => generated 385 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 10:31:59.896267 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131b6bf4-6c02-49cf-89b8-beed5ae8d38f admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:31:59.939284 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 972/1930] 10.222.0.49 () {58 vars in 957 bytes} [Wed Jun 15 10:31:59 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:31:59.950627 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a13eb2-94e3-4222-b4fa-ea251a58483d None None] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:31:59.952231 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-60a13eb2-94e3-4222-b4fa-ea251a58483d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:31:59.992031 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131b6bf4-6c02-49cf-89b8-beed5ae8d38f admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:31:59.998767 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8f97b4-802a-48a4-bb3c-9eca37f09dd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:31:59.999635 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8f97b4-802a-48a4-bb3c-9eca37f09dd6 admin admin] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:32:00.000609 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8f97b4-802a-48a4-bb3c-9eca37f09dd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:32:00.001016 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8f97b4-802a-48a4-bb3c-9eca37f09dd6 admin admin] PATH_INFO: `/v3/users` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:32:00.001716 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a8f97b4-802a-48a4-bb3c-9eca37f09dd6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:32:00.003669 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a8f97b4-802a-48a4-bb3c-9eca37f09dd6 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:32:00.029476 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 959/1931] 10.222.0.49 () {60 vars in 1143 bytes} [Wed Jun 15 10:31:59 2022] POST /identity/v3/users => generated 293 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 10:32:00.035569 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8f97b4-802a-48a4-bb3c-9eca37f09dd6 admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:32:00.041141 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a13eb2-94e3-4222-b4fa-ea251a58483d None None] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:32:00.047910 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8f97b4-802a-48a4-bb3c-9eca37f09dd6 admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:32:00.057096 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d85139-99cb-4c38-86f4-66519484dc6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:32:00.058325 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d85139-99cb-4c38-86f4-66519484dc6d admin admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:32:00.058685 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d85139-99cb-4c38-86f4-66519484dc6d admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:32:00.059401 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d85139-99cb-4c38-86f4-66519484dc6d admin admin] PATH_INFO: `/v3/roles` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:32:00.060152 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d85139-99cb-4c38-86f4-66519484dc6d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:32:00.062094 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d85139-99cb-4c38-86f4-66519484dc6d admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:32:00.070729 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=100 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:32:00.077708 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 960/1932] 10.222.0.49 () {58 vars in 1121 bytes} [Wed Jun 15 10:32:00 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 15 10:32:00.078624 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d85139-99cb-4c38-86f4-66519484dc6d admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:32:00.093948 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d85139-99cb-4c38-86f4-66519484dc6d admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:32:00.100204 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a29675-2a93-45f0-8618-6a1645399332 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:32:00.100893 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a29675-2a93-45f0-8618-6a1645399332 admin admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:32:00.101397 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a29675-2a93-45f0-8618-6a1645399332 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:32:00.102323 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a29675-2a93-45f0-8618-6a1645399332 admin admin] PATH_INFO: `/v3/roles` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:32:00.103235 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a29675-2a93-45f0-8618-6a1645399332 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:32:00.105140 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a29675-2a93-45f0-8618-6a1645399332 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:32:00.113562 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92042311-3373-4842-a827-22c86ea34cac service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20d1327553404b80aecee87c78fcb52b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '559b3e713f5844b6ae37151d56dd419f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:32:00.114427 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92042311-3373-4842-a827-22c86ea34cac service cinder] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:32:00.114535 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92042311-3373-4842-a827-22c86ea34cac service cinder] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:32:00.114824 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92042311-3373-4842-a827-22c86ea34cac service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:32:00.123978 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 961/1933] 10.222.0.49 () {58 vars in 1121 bytes} [Wed Jun 15 10:32:00 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 10:32:00.134857 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a29675-2a93-45f0-8618-6a1645399332 admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:32:00.162435 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a29675-2a93-45f0-8618-6a1645399332 admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:32:00.166079 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9f9cff-9bac-4d60-91ef-4afe0f35e85d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:32:00.167217 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9f9cff-9bac-4d60-91ef-4afe0f35e85d admin admin] REQUEST_METHOD: `PUT` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:32:00.167893 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9f9cff-9bac-4d60-91ef-4afe0f35e85d admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:32:00.168152 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9f9cff-9bac-4d60-91ef-4afe0f35e85d admin admin] PATH_INFO: `/v3/projects/3294948606364d24bc5beb0d13eb51b0/users/19dbc90156e74a50aa55d9c9f13bbe4d/roles/6876e57cc51c448097ab109d35b2a3da` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:32:00.188764 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e9f9cff-9bac-4d60-91ef-4afe0f35e85d admin admin] RBAC: Authorizing `identity:create_grant(project_id=3294948606364d24bc5beb0d13eb51b0, user_id=19dbc90156e74a50aa55d9c9f13bbe4d, role_id=6876e57cc51c448097ab109d35b2a3da)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:32:00.190427 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e9f9cff-9bac-4d60-91ef-4afe0f35e85d admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:32:00.197607 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92042311-3373-4842-a827-22c86ea34cac service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:32:00.199327 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92042311-3373-4842-a827-22c86ea34cac service cinder] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:32:00.211631 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 973/1934] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Jun 15 10:31:59 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 259 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 10:32:00.212088 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 962/1935] 10.222.0.49 () {60 vars in 1482 bytes} [Wed Jun 15 10:32:00 2022] PUT /identity/v3/projects/3294948606364d24bc5beb0d13eb51b0/users/19dbc90156e74a50aa55d9c9f13bbe4d/roles/6876e57cc51c448097ab109d35b2a3da => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:32:00.221269 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0571733-68bf-48dc-9664-c932d2bd0d4a None None] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:32:00.221556 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0571733-68bf-48dc-9664-c932d2bd0d4a None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:32:00.221836 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0571733-68bf-48dc-9664-c932d2bd0d4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:32:00.257447 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.auth.core [None req-a0571733-68bf-48dc-9664-c932d2bd0d4a None None] MFA Rules not processed for user `19dbc90156e74a50aa55d9c9f13bbe4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:32:00.258776 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-a0571733-68bf-48dc-9664-c932d2bd0d4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:32:00.368671 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9f9cff-9bac-4d60-91ef-4afe0f35e85d admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:32:00.382239 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 974/1936] 10.222.0.49 () {58 vars in 957 bytes} [Wed Jun 15 10:32:00 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:32:00.483817 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9f9cff-9bac-4d60-91ef-4afe0f35e85d admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:32:00.490436 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff82faa3-376d-4e09-9c35-be98cd110dd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:32:00.491422 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff82faa3-376d-4e09-9c35-be98cd110dd0 admin admin] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:32:00.491941 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff82faa3-376d-4e09-9c35-be98cd110dd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:32:00.492566 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff82faa3-376d-4e09-9c35-be98cd110dd0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:32:00.493684 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff82faa3-376d-4e09-9c35-be98cd110dd0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:32:00.496062 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff82faa3-376d-4e09-9c35-be98cd110dd0 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:32:00.500172 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:32:00.535129 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 963/1937] 10.222.0.49 () {60 vars in 1152 bytes} [Wed Jun 15 10:32:00 2022] POST /identity/v3/projects => generated 387 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 10:32:00.537897 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0571733-68bf-48dc-9664-c932d2bd0d4a None None] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:32:00.552293 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0571733-68bf-48dc-9664-c932d2bd0d4a None None] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:32:00.614791 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc628c77-2b73-4e0c-a145-f0de2f46c366 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:32:00.615507 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc628c77-2b73-4e0c-a145-f0de2f46c366 admin admin] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:32:00.615791 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc628c77-2b73-4e0c-a145-f0de2f46c366 admin admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:32:00.616072 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc628c77-2b73-4e0c-a145-f0de2f46c366 admin admin] PATH_INFO: `/v3/users` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:32:00.616835 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc628c77-2b73-4e0c-a145-f0de2f46c366 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:32:00.618792 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc628c77-2b73-4e0c-a145-f0de2f46c366 admin admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:32:00.626451 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:32:00.641819 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 975/1938] 10.222.0.49 () {60 vars in 1143 bytes} [Wed Jun 15 10:32:00 2022] POST /identity/v3/users => generated 294 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 10:32:00.645739 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff82faa3-376d-4e09-9c35-be98cd110dd0 admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:32:00.657050 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff82faa3-376d-4e09-9c35-be98cd110dd0 admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:32:00.663721 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a13437e-554d-479e-a422-92d4ba560fc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:32:00.663721 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a13437e-554d-479e-a422-92d4ba560fc4 admin admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:32:00.663721 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a13437e-554d-479e-a422-92d4ba560fc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:32:00.663721 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a13437e-554d-479e-a422-92d4ba560fc4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:32:00.664848 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a13437e-554d-479e-a422-92d4ba560fc4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:32:00.666359 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a13437e-554d-479e-a422-92d4ba560fc4 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:32:00.679878 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 964/1939] 10.222.0.49 () {58 vars in 1121 bytes} [Wed Jun 15 10:32:00 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 15 10:32:00.683762 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc628c77-2b73-4e0c-a145-f0de2f46c366 admin admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:32:00.694254 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc628c77-2b73-4e0c-a145-f0de2f46c366 admin admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:32:00.699930 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d92367-1b92-4a04-b07d-d4aec992f4bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:32:00.700609 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d92367-1b92-4a04-b07d-d4aec992f4bc admin admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:32:00.700892 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d92367-1b92-4a04-b07d-d4aec992f4bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:32:00.701165 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d92367-1b92-4a04-b07d-d4aec992f4bc admin admin] PATH_INFO: `/v3/roles` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:32:00.703204 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d92367-1b92-4a04-b07d-d4aec992f4bc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:32:00.704709 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d92367-1b92-4a04-b07d-d4aec992f4bc admin admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:32:00.718438 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 976/1940] 10.222.0.49 () {58 vars in 1121 bytes} [Wed Jun 15 10:32:00 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 10:32:00.722667 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a13437e-554d-479e-a422-92d4ba560fc4 admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:32:00.736669 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d92367-1b92-4a04-b07d-d4aec992f4bc admin admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:32:00.738264 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a13437e-554d-479e-a422-92d4ba560fc4 admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:32:00.743516 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a0c1b1-0f47-4660-b005-52dadb346e34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:32:00.744105 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a0c1b1-0f47-4660-b005-52dadb346e34 admin admin] REQUEST_METHOD: `PUT` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:32:00.744383 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a0c1b1-0f47-4660-b005-52dadb346e34 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:32:00.744656 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a0c1b1-0f47-4660-b005-52dadb346e34 admin admin] PATH_INFO: `/v3/projects/bc59f64c4b7649ac9ba1934c64e5ce60/users/91b75c5f75f24de08481f4fcefdbac7b/roles/6876e57cc51c448097ab109d35b2a3da` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:32:00.747952 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d92367-1b92-4a04-b07d-d4aec992f4bc admin admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:32:00.752806 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda8c983-c089-49b0-b3de-3fdb8aaae883 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:32:00.753680 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda8c983-c089-49b0-b3de-3fdb8aaae883 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:32:00.753962 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda8c983-c089-49b0-b3de-3fdb8aaae883 admin admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:32:00.754265 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda8c983-c089-49b0-b3de-3fdb8aaae883 admin admin] PATH_INFO: `/v3/users/8ca46e4a0945427a89b6944d5004cafc` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:32:00.755898 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda8c983-c089-49b0-b3de-3fdb8aaae883 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8ca46e4a0945427a89b6944d5004cafc)` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:32:00.758435 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda8c983-c089-49b0-b3de-3fdb8aaae883 admin admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:32:00.766212 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a0c1b1-0f47-4660-b005-52dadb346e34 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bc59f64c4b7649ac9ba1934c64e5ce60, user_id=91b75c5f75f24de08481f4fcefdbac7b, role_id=6876e57cc51c448097ab109d35b2a3da)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:32:00.768978 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a0c1b1-0f47-4660-b005-52dadb346e34 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:32:00.787752 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 965/1941] 10.222.0.49 () {60 vars in 1482 bytes} [Wed Jun 15 10:32:00 2022] PUT /identity/v3/projects/bc59f64c4b7649ac9ba1934c64e5ce60/users/91b75c5f75f24de08481f4fcefdbac7b/roles/6876e57cc51c448097ab109d35b2a3da => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:32:00.799768 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a21f037-f3fd-4781-8300-de64a7261b88 None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:32:00.800130 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a21f037-f3fd-4781-8300-de64a7261b88 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:32:00.800408 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a21f037-f3fd-4781-8300-de64a7261b88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:32:00.818956 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-cda8c983-c089-49b0-b3de-3fdb8aaae883 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8ca46e4a0945427a89b6944d5004cafc'} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:32:00.827249 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-cda8c983-c089-49b0-b3de-3fdb8aaae883 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8ca46e4a0945427a89b6944d5004cafc'} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:32:00.828130 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-cda8c983-c089-49b0-b3de-3fdb8aaae883 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8ca46e4a0945427a89b6944d5004cafc'} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:32:00.828768 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-cda8c983-c089-49b0-b3de-3fdb8aaae883 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8ca46e4a0945427a89b6944d5004cafc'} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:32:00.834008 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-3a21f037-f3fd-4781-8300-de64a7261b88 None None] MFA Rules not processed for user `91b75c5f75f24de08481f4fcefdbac7b`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:32:00.835619 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-3a21f037-f3fd-4781-8300-de64a7261b88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:32:00.838727 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-cda8c983-c089-49b0-b3de-3fdb8aaae883 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8ca46e4a0945427a89b6944d5004cafc'} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:32:00.857843 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 977/1942] 10.222.0.49 () {58 vars in 1223 bytes} [Wed Jun 15 10:32:00 2022] DELETE /identity/v3/users/8ca46e4a0945427a89b6944d5004cafc => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 10:32:00.862700 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7090dd12-f28f-4c87-873c-828cd245e8a3 None None] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:32:00.863010 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7090dd12-f28f-4c87-873c-828cd245e8a3 None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:32:00.863314 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7090dd12-f28f-4c87-873c-828cd245e8a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:32:00.881668 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.auth.core [None req-7090dd12-f28f-4c87-873c-828cd245e8a3 None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:32:00.883445 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-7090dd12-f28f-4c87-873c-828cd245e8a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:32:00.903509 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 966/1943] 10.222.0.49 () {58 vars in 957 bytes} [Wed Jun 15 10:32:00 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:32:00.908194 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a21f037-f3fd-4781-8300-de64a7261b88 None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:32:00.910866 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-3a21f037-f3fd-4781-8300-de64a7261b88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:32:00.941927 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 978/1944] 10.222.0.49 () {58 vars in 957 bytes} [Wed Jun 15 10:32:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:32:00.952136 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7090dd12-f28f-4c87-873c-828cd245e8a3 None None] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:32:00.953509 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-7090dd12-f28f-4c87-873c-828cd245e8a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:32:00.975934 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a21f037-f3fd-4781-8300-de64a7261b88 None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:32:00.981072 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943c156f-db11-4555-875a-a50c21760bf4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:32:00.981731 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943c156f-db11-4555-875a-a50c21760bf4 admin admin] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:32:00.982014 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943c156f-db11-4555-875a-a50c21760bf4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:32:00.982290 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943c156f-db11-4555-875a-a50c21760bf4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:32:00.982977 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943c156f-db11-4555-875a-a50c21760bf4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:32:00.984890 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943c156f-db11-4555-875a-a50c21760bf4 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:32:01.009866 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7090dd12-f28f-4c87-873c-828cd245e8a3 None None] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:32:01.011133 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 967/1945] 10.222.0.49 () {60 vars in 1152 bytes} [Wed Jun 15 10:32:00 2022] POST /identity/v3/projects => generated 387 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 10:32:01.014920 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943c156f-db11-4555-875a-a50c21760bf4 admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:32:01.017545 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:INSERT=3,keystone:SELECT=87,keystone:DELETE=10 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:32:01.042175 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:32:01.066028 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba9c230-4029-4c83-a76b-5178b7bc0591 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0cb44a17d205431793e9a8791e477a32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '559b3e713f5844b6ae37151d56dd419f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:32:01.066465 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba9c230-4029-4c83-a76b-5178b7bc0591 service neutron] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:32:01.066757 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba9c230-4029-4c83-a76b-5178b7bc0591 service neutron] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:32:01.067040 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba9c230-4029-4c83-a76b-5178b7bc0591 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:32:01.079992 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943c156f-db11-4555-875a-a50c21760bf4 admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:32:01.084919 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e6015f-a8df-45ec-a6df-1bd377695e59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:32:01.085622 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e6015f-a8df-45ec-a6df-1bd377695e59 admin admin] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:32:01.085948 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e6015f-a8df-45ec-a6df-1bd377695e59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:32:01.086233 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e6015f-a8df-45ec-a6df-1bd377695e59 admin admin] PATH_INFO: `/v3/users` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:32:01.086990 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e6015f-a8df-45ec-a6df-1bd377695e59 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:32:01.088960 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e6015f-a8df-45ec-a6df-1bd377695e59 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:32:01.113116 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 968/1946] 10.222.0.49 () {60 vars in 1143 bytes} [Wed Jun 15 10:32:01 2022] POST /identity/v3/users => generated 300 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 10:32:01.114707 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e6015f-a8df-45ec-a6df-1bd377695e59 admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:32:01.129670 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e6015f-a8df-45ec-a6df-1bd377695e59 admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:32:01.134994 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba9c230-4029-4c83-a76b-5178b7bc0591 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:32:01.136423 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a5a672-f995-4f16-891e-0232cff2b259 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:32:01.136615 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba9c230-4029-4c83-a76b-5178b7bc0591 service neutron] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:32:01.137228 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a5a672-f995-4f16-891e-0232cff2b259 admin admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:32:01.137581 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a5a672-f995-4f16-891e-0232cff2b259 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:32:01.137925 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a5a672-f995-4f16-891e-0232cff2b259 admin admin] PATH_INFO: `/v3/roles` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:32:01.138762 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a5a672-f995-4f16-891e-0232cff2b259 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:32:01.140625 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a5a672-f995-4f16-891e-0232cff2b259 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:32:01.147829 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 979/1947] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Jun 15 10:32:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 196 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 10:32:01.151688 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 969/1948] 10.222.0.49 () {58 vars in 1121 bytes} [Wed Jun 15 10:32:01 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 10:32:01.155995 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba9c230-4029-4c83-a76b-5178b7bc0591 service neutron] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:32:01.166241 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba9c230-4029-4c83-a76b-5178b7bc0591 service neutron] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:32:01.170987 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3ef44b-bbc2-4f16-a0e3-3311cfa96801 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:32:01.171933 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3ef44b-bbc2-4f16-a0e3-3311cfa96801 admin admin] REQUEST_METHOD: `PUT` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:32:01.172219 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3ef44b-bbc2-4f16-a0e3-3311cfa96801 admin admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:32:01.172503 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3ef44b-bbc2-4f16-a0e3-3311cfa96801 admin admin] PATH_INFO: `/v3/projects/965147f94aa84a8ca784939a9d5aafbc/users/cf66f62d7d664e1d8e4035a465934fd4/roles/8051174e7a7947d5b991ebd793c2e543` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:32:01.198327 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d3ef44b-bbc2-4f16-a0e3-3311cfa96801 admin admin] RBAC: Authorizing `identity:create_grant(project_id=965147f94aa84a8ca784939a9d5aafbc, user_id=cf66f62d7d664e1d8e4035a465934fd4, role_id=8051174e7a7947d5b991ebd793c2e543)` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:32:01.201656 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d3ef44b-bbc2-4f16-a0e3-3311cfa96801 admin admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:32:01.222900 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 980/1949] 10.222.0.49 () {60 vars in 1482 bytes} [Wed Jun 15 10:32:01 2022] PUT /identity/v3/projects/965147f94aa84a8ca784939a9d5aafbc/users/cf66f62d7d664e1d8e4035a465934fd4/roles/8051174e7a7947d5b991ebd793c2e543 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 10:32:01.223888 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a5a672-f995-4f16-891e-0232cff2b259 admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:32:01.242868 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a5a672-f995-4f16-891e-0232cff2b259 admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:32:01.318569 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a058eeb-9dd1-43a6-a55d-155c145faa7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:32:01.319285 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a058eeb-9dd1-43a6-a55d-155c145faa7a admin admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:32:01.319628 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a058eeb-9dd1-43a6-a55d-155c145faa7a admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:32:01.319844 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a058eeb-9dd1-43a6-a55d-155c145faa7a admin admin] PATH_INFO: `/v3/roles` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:32:01.321231 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a058eeb-9dd1-43a6-a55d-155c145faa7a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:32:01.322912 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a058eeb-9dd1-43a6-a55d-155c145faa7a admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:32:01.333650 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 970/1950] 10.222.0.49 () {58 vars in 1121 bytes} [Wed Jun 15 10:32:01 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 10:32:01.337923 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3ef44b-bbc2-4f16-a0e3-3311cfa96801 admin admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:32:01.348178 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3ef44b-bbc2-4f16-a0e3-3311cfa96801 admin admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:32:01.352802 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39b1c35-5782-496c-8780-b47ebb56aa61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:32:01.353692 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39b1c35-5782-496c-8780-b47ebb56aa61 admin admin] REQUEST_METHOD: `PUT` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:32:01.353976 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39b1c35-5782-496c-8780-b47ebb56aa61 admin admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:32:01.354250 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39b1c35-5782-496c-8780-b47ebb56aa61 admin admin] PATH_INFO: `/v3/projects/965147f94aa84a8ca784939a9d5aafbc/users/cf66f62d7d664e1d8e4035a465934fd4/roles/8051174e7a7947d5b991ebd793c2e543` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:32:01.356239 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f39b1c35-5782-496c-8780-b47ebb56aa61 admin admin] RBAC: Authorizing `identity:create_grant(project_id=965147f94aa84a8ca784939a9d5aafbc, user_id=cf66f62d7d664e1d8e4035a465934fd4, role_id=8051174e7a7947d5b991ebd793c2e543)` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:32:01.358868 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f39b1c35-5782-496c-8780-b47ebb56aa61 admin admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:32:01.369777 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 981/1951] 10.222.0.49 () {60 vars in 1482 bytes} [Wed Jun 15 10:32:01 2022] PUT /identity/v3/projects/965147f94aa84a8ca784939a9d5aafbc/users/cf66f62d7d664e1d8e4035a465934fd4/roles/8051174e7a7947d5b991ebd793c2e543 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:32:01.379992 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e81f1a3-2943-418e-bfe9-91333997bd52 None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:32:01.380290 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e81f1a3-2943-418e-bfe9-91333997bd52 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:32:01.380569 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e81f1a3-2943-418e-bfe9-91333997bd52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:32:01.420541 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-1e81f1a3-2943-418e-bfe9-91333997bd52 None None] MFA Rules not processed for user `cf66f62d7d664e1d8e4035a465934fd4`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:32:01.422053 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-1e81f1a3-2943-418e-bfe9-91333997bd52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:32:01.446612 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39b1c35-5782-496c-8780-b47ebb56aa61 admin admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:32:01.448575 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-f39b1c35-5782-496c-8780-b47ebb56aa61 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:32:01.503906 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 971/1952] 10.222.0.49 () {58 vars in 957 bytes} [Wed Jun 15 10:32:01 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:32:01.515169 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e81f1a3-2943-418e-bfe9-91333997bd52 None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:32:01.516475 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-1e81f1a3-2943-418e-bfe9-91333997bd52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:32:01.522544 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39b1c35-5782-496c-8780-b47ebb56aa61 admin admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:32:01.527411 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7356601a-990f-45c9-a717-371b9903b96a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:32:01.528279 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7356601a-990f-45c9-a717-371b9903b96a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:32:01.528890 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7356601a-990f-45c9-a717-371b9903b96a admin admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:32:01.528890 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7356601a-990f-45c9-a717-371b9903b96a admin admin] PATH_INFO: `/v3/projects/672f282e93b3475e95c5e4bc9a8d2ab9` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:32:01.530457 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7356601a-990f-45c9-a717-371b9903b96a admin admin] RBAC: Authorizing `identity:delete_project(project_id=672f282e93b3475e95c5e4bc9a8d2ab9)` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:32:01.533905 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7356601a-990f-45c9-a717-371b9903b96a admin admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:32:01.551714 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:32:01.563050 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=97,keystone:INSERT=2,keystone:DELETE=1 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:32:01.588600 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e81f1a3-2943-418e-bfe9-91333997bd52 None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:32:01.591618 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-7356601a-990f-45c9-a717-371b9903b96a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '672f282e93b3475e95c5e4bc9a8d2ab9', 'request_id': 'req-7356601a-990f-45c9-a717-371b9903b96a'} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:32:01.603690 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-7356601a-990f-45c9-a717-371b9903b96a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '672f282e93b3475e95c5e4bc9a8d2ab9', 'request_id': 'req-7356601a-990f-45c9-a717-371b9903b96a'} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:32:01.611814 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-7356601a-990f-45c9-a717-371b9903b96a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '672f282e93b3475e95c5e4bc9a8d2ab9', 'request_id': 'req-7356601a-990f-45c9-a717-371b9903b96a'} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:32:01.623765 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-7356601a-990f-45c9-a717-371b9903b96a admin admin] The token cache is being invalidate because project 672f282e93b3475e95c5e4bc9a8d2ab9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80119) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 10:32:01.623996 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-7356601a-990f-45c9-a717-371b9903b96a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:32:01.624717 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-7356601a-990f-45c9-a717-371b9903b96a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:32:01.626602 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 982/1953] 10.222.0.49 () {58 vars in 1232 bytes} [Wed Jun 15 10:32:01 2022] DELETE /identity/v3/projects/672f282e93b3475e95c5e4bc9a8d2ab9 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 10:32:01.629949 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7356601a-990f-45c9-a717-371b9903b96a admin admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:32:01.631307 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-7356601a-990f-45c9-a717-371b9903b96a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:32:01.649965 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde770a4-2678-45c2-8ed1-7b4700fdbb38 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20d1327553404b80aecee87c78fcb52b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '559b3e713f5844b6ae37151d56dd419f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:32:01.650628 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde770a4-2678-45c2-8ed1-7b4700fdbb38 service cinder] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:32:01.650912 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde770a4-2678-45c2-8ed1-7b4700fdbb38 service cinder] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:32:01.651224 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde770a4-2678-45c2-8ed1-7b4700fdbb38 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:32:01.654191 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-dde770a4-2678-45c2-8ed1-7b4700fdbb38 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:32:01.702499 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7356601a-990f-45c9-a717-371b9903b96a admin admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:32:01.703726 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856e9964-d274-479a-89f4-6c5f64a1a308 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:32:01.704595 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856e9964-d274-479a-89f4-6c5f64a1a308 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:32:01.704877 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856e9964-d274-479a-89f4-6c5f64a1a308 admin admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:32:01.705154 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856e9964-d274-479a-89f4-6c5f64a1a308 admin admin] PATH_INFO: `/v3/users/916b4b25302a4415b00a111f6a53f606` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:32:01.706343 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-856e9964-d274-479a-89f4-6c5f64a1a308 admin admin] RBAC: Authorizing `identity:delete_user(user_id=916b4b25302a4415b00a111f6a53f606)` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:32:01.708286 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-856e9964-d274-479a-89f4-6c5f64a1a308 admin admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:32:01.724124 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde770a4-2678-45c2-8ed1-7b4700fdbb38 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:32:01.725556 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde770a4-2678-45c2-8ed1-7b4700fdbb38 service cinder] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:32:01.736649 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 972/1954] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Jun 15 10:32:01 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 222 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 10:32:01.757436 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-856e9964-d274-479a-89f4-6c5f64a1a308 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '916b4b25302a4415b00a111f6a53f606'} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:32:01.764476 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-856e9964-d274-479a-89f4-6c5f64a1a308 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '916b4b25302a4415b00a111f6a53f606'} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:32:01.765229 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-856e9964-d274-479a-89f4-6c5f64a1a308 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '916b4b25302a4415b00a111f6a53f606'} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:32:01.765806 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-856e9964-d274-479a-89f4-6c5f64a1a308 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '916b4b25302a4415b00a111f6a53f606'} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:32:01.774241 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-856e9964-d274-479a-89f4-6c5f64a1a308 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '916b4b25302a4415b00a111f6a53f606'} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:32:01.796744 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 983/1955] 10.222.0.49 () {58 vars in 1223 bytes} [Wed Jun 15 10:32:01 2022] DELETE /identity/v3/users/916b4b25302a4415b00a111f6a53f606 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:32:02.160989 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde770a4-2678-45c2-8ed1-7b4700fdbb38 service cinder] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:32:02.178168 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde770a4-2678-45c2-8ed1-7b4700fdbb38 service cinder] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:32:02.258651 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9276474-d7a2-4e47-bb52-6724b905a39c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:32:02.259571 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9276474-d7a2-4e47-bb52-6724b905a39c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:32:02.259869 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9276474-d7a2-4e47-bb52-6724b905a39c admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:32:02.260137 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9276474-d7a2-4e47-bb52-6724b905a39c admin admin] PATH_INFO: `/v3/projects/1e7d23ac4f2a4aba974e163d63d9d85b` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:32:02.261636 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9276474-d7a2-4e47-bb52-6724b905a39c admin admin] RBAC: Authorizing `identity:delete_project(project_id=1e7d23ac4f2a4aba974e163d63d9d85b)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:32:02.264306 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9276474-d7a2-4e47-bb52-6724b905a39c admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:32:02.313541 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:32:02.336910 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-d9276474-d7a2-4e47-bb52-6724b905a39c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1e7d23ac4f2a4aba974e163d63d9d85b', 'request_id': 'req-d9276474-d7a2-4e47-bb52-6724b905a39c'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:32:02.349981 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-d9276474-d7a2-4e47-bb52-6724b905a39c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1e7d23ac4f2a4aba974e163d63d9d85b', 'request_id': 'req-d9276474-d7a2-4e47-bb52-6724b905a39c'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:32:02.356590 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-d9276474-d7a2-4e47-bb52-6724b905a39c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1e7d23ac4f2a4aba974e163d63d9d85b', 'request_id': 'req-d9276474-d7a2-4e47-bb52-6724b905a39c'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:32:02.369193 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-d9276474-d7a2-4e47-bb52-6724b905a39c admin admin] The token cache is being invalidate because project 1e7d23ac4f2a4aba974e163d63d9d85b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80120) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 10:32:02.369484 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-d9276474-d7a2-4e47-bb52-6724b905a39c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:32:02.372935 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-d9276474-d7a2-4e47-bb52-6724b905a39c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:32:02.375533 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 973/1956] 10.222.0.49 () {58 vars in 1232 bytes} [Wed Jun 15 10:32:02 2022] DELETE /identity/v3/projects/1e7d23ac4f2a4aba974e163d63d9d85b => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:32:02.384998 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4044df9-8b49-4343-b3c1-b59d46d1ce6e None None] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:32:02.385539 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4044df9-8b49-4343-b3c1-b59d46d1ce6e None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:32:02.385539 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4044df9-8b49-4343-b3c1-b59d46d1ce6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:32:02.395179 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9276474-d7a2-4e47-bb52-6724b905a39c admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:32:02.397358 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-d9276474-d7a2-4e47-bb52-6724b905a39c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:32:02.413048 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.auth.core [None req-a4044df9-8b49-4343-b3c1-b59d46d1ce6e None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:32:02.414637 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-a4044df9-8b49-4343-b3c1-b59d46d1ce6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:32:02.457059 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:32:02.501333 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9276474-d7a2-4e47-bb52-6724b905a39c admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:32:02.502680 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 984/1957] 10.222.0.49 () {58 vars in 957 bytes} [Wed Jun 15 10:32:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:32:02.511271 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9da681-9f67-47de-9ff6-f1f669b405c1 None None] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:32:02.511617 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9da681-9f67-47de-9ff6-f1f669b405c1 None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:32:02.511937 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9da681-9f67-47de-9ff6-f1f669b405c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:32:02.512175 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5480314f-b94c-4bea-9721-5b9523937dcd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20d1327553404b80aecee87c78fcb52b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '559b3e713f5844b6ae37151d56dd419f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:32:02.512841 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5480314f-b94c-4bea-9721-5b9523937dcd service cinder] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:32:02.513130 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5480314f-b94c-4bea-9721-5b9523937dcd service cinder] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:32:02.513410 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5480314f-b94c-4bea-9721-5b9523937dcd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:32:02.515561 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-5480314f-b94c-4bea-9721-5b9523937dcd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:32:02.534741 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.auth.core [None req-bd9da681-9f67-47de-9ff6-f1f669b405c1 None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:32:02.536201 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-bd9da681-9f67-47de-9ff6-f1f669b405c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:32:02.550201 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 985/1958] 10.222.0.49 () {58 vars in 957 bytes} [Wed Jun 15 10:32:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 10:32:02.555539 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9da681-9f67-47de-9ff6-f1f669b405c1 None None] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:32:02.565950 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9da681-9f67-47de-9ff6-f1f669b405c1 None None] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:32:02.571899 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546671bc-1eb6-4487-b727-af6b7bbde3f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:32:02.572604 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546671bc-1eb6-4487-b727-af6b7bbde3f6 admin admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:32:02.572879 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546671bc-1eb6-4487-b727-af6b7bbde3f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:32:02.573343 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546671bc-1eb6-4487-b727-af6b7bbde3f6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:32:02.574014 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-546671bc-1eb6-4487-b727-af6b7bbde3f6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:32:02.574321 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-546671bc-1eb6-4487-b727-af6b7bbde3f6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:32:02.576086 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-546671bc-1eb6-4487-b727-af6b7bbde3f6 admin admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:32:02.591082 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 986/1959] 10.222.0.49 () {58 vars in 1152 bytes} [Wed Jun 15 10:32:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 10:32:02.596160 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546671bc-1eb6-4487-b727-af6b7bbde3f6 admin admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:32:02.604777 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5480314f-b94c-4bea-9721-5b9523937dcd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:32:02.606256 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5480314f-b94c-4bea-9721-5b9523937dcd service cinder] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:32:02.612853 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546671bc-1eb6-4487-b727-af6b7bbde3f6 admin admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:32:02.615851 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 974/1960] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Jun 15 10:32:02 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 221 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:32:02.620235 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997a578e-50eb-426b-ba45-351e1d5e7455 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:32:02.621037 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997a578e-50eb-426b-ba45-351e1d5e7455 admin admin] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:32:02.621689 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997a578e-50eb-426b-ba45-351e1d5e7455 admin admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:32:02.621689 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997a578e-50eb-426b-ba45-351e1d5e7455 admin admin] PATH_INFO: `/v3/projects` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:32:02.622284 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997a578e-50eb-426b-ba45-351e1d5e7455 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:32:02.624263 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997a578e-50eb-426b-ba45-351e1d5e7455 admin admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:32:02.653237 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 987/1961] 10.222.0.49 () {60 vars in 1152 bytes} [Wed Jun 15 10:32:02 2022] POST /identity/v3/projects => generated 377 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 10:32:02.657772 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5480314f-b94c-4bea-9721-5b9523937dcd service cinder] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:32:02.668334 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5480314f-b94c-4bea-9721-5b9523937dcd service cinder] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:32:02.733984 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa799580-0c31-45b8-99c6-3d84ee571b83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:32:02.734670 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa799580-0c31-45b8-99c6-3d84ee571b83 admin admin] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:32:02.735163 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa799580-0c31-45b8-99c6-3d84ee571b83 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:32:02.735290 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa799580-0c31-45b8-99c6-3d84ee571b83 admin admin] PATH_INFO: `/v3/users` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:32:02.735950 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa799580-0c31-45b8-99c6-3d84ee571b83 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:32:02.738482 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa799580-0c31-45b8-99c6-3d84ee571b83 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:32:02.765360 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 975/1962] 10.222.0.49 () {60 vars in 1143 bytes} [Wed Jun 15 10:32:02 2022] POST /identity/v3/users => generated 289 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 10:32:02.769819 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997a578e-50eb-426b-ba45-351e1d5e7455 admin admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:32:02.794705 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997a578e-50eb-426b-ba45-351e1d5e7455 admin admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:32:02.800995 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b373975-8780-49bd-80c5-dad20c8da880 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:32:02.801670 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b373975-8780-49bd-80c5-dad20c8da880 admin admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:32:02.801956 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b373975-8780-49bd-80c5-dad20c8da880 admin admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:32:02.802242 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b373975-8780-49bd-80c5-dad20c8da880 admin admin] PATH_INFO: `/v3/roles` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:32:02.802951 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b373975-8780-49bd-80c5-dad20c8da880 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:32:02.805146 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b373975-8780-49bd-80c5-dad20c8da880 admin admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:32:02.817656 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 988/1963] 10.222.0.49 () {58 vars in 1121 bytes} [Wed Jun 15 10:32:02 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 10:32:02.822168 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa799580-0c31-45b8-99c6-3d84ee571b83 admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:32:02.830196 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:32:02.834240 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa799580-0c31-45b8-99c6-3d84ee571b83 admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:32:02.840482 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e416c19-574c-4ebe-adf9-1bafa8846f22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:32:02.841071 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e416c19-574c-4ebe-adf9-1bafa8846f22 admin admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:32:02.841587 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e416c19-574c-4ebe-adf9-1bafa8846f22 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:32:02.843259 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e416c19-574c-4ebe-adf9-1bafa8846f22 admin admin] PATH_INFO: `/v3/roles` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:32:02.844091 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e416c19-574c-4ebe-adf9-1bafa8846f22 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:32:02.845936 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e416c19-574c-4ebe-adf9-1bafa8846f22 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:32:02.859716 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 976/1964] 10.222.0.49 () {58 vars in 1121 bytes} [Wed Jun 15 10:32:02 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 15 10:32:02.863939 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b373975-8780-49bd-80c5-dad20c8da880 admin admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:32:02.877192 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b373975-8780-49bd-80c5-dad20c8da880 admin admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:32:02.883867 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3869a92-8a45-4fb6-8fc7-660435175926 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:32:02.884762 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3869a92-8a45-4fb6-8fc7-660435175926 admin admin] REQUEST_METHOD: `PUT` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:32:02.885878 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3869a92-8a45-4fb6-8fc7-660435175926 admin admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:32:02.886167 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3869a92-8a45-4fb6-8fc7-660435175926 admin admin] PATH_INFO: `/v3/projects/3e00222d3870456187378cdc6ccb499a/users/320d3accb5ae426387f8f46928473ab0/roles/6876e57cc51c448097ab109d35b2a3da` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:32:02.903717 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3869a92-8a45-4fb6-8fc7-660435175926 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3e00222d3870456187378cdc6ccb499a, user_id=320d3accb5ae426387f8f46928473ab0, role_id=6876e57cc51c448097ab109d35b2a3da)` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:32:02.906397 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3869a92-8a45-4fb6-8fc7-660435175926 admin admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:32:02.926500 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 989/1965] 10.222.0.49 () {60 vars in 1482 bytes} [Wed Jun 15 10:32:02 2022] PUT /identity/v3/projects/3e00222d3870456187378cdc6ccb499a/users/320d3accb5ae426387f8f46928473ab0/roles/6876e57cc51c448097ab109d35b2a3da => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:32:02.938527 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ca0d99-240c-4cbb-ac8f-5d14753ad424 None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:32:02.938846 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ca0d99-240c-4cbb-ac8f-5d14753ad424 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:32:02.939368 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ca0d99-240c-4cbb-ac8f-5d14753ad424 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:32:02.981277 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-78ca0d99-240c-4cbb-ac8f-5d14753ad424 None None] MFA Rules not processed for user `320d3accb5ae426387f8f46928473ab0`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:32:02.982542 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-78ca0d99-240c-4cbb-ac8f-5d14753ad424 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:32:03.051485 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 977/1966] 10.222.0.49 () {58 vars in 957 bytes} [Wed Jun 15 10:32:02 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:32:03.063390 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3869a92-8a45-4fb6-8fc7-660435175926 admin admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:32:03.065056 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-c3869a92-8a45-4fb6-8fc7-660435175926 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:32:03.127848 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3869a92-8a45-4fb6-8fc7-660435175926 admin admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:32:03.131591 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebf76e0-3c27-4995-85ca-b5b45002f130 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91f2fe3be62e49548eae208e57bf2a93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '559b3e713f5844b6ae37151d56dd419f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:32:03.132247 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebf76e0-3c27-4995-85ca-b5b45002f130 service glance] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:32:03.132532 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebf76e0-3c27-4995-85ca-b5b45002f130 service glance] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:32:03.132861 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebf76e0-3c27-4995-85ca-b5b45002f130 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:32:03.144033 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ebf76e0-3c27-4995-85ca-b5b45002f130 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:32:03.145474 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ebf76e0-3c27-4995-85ca-b5b45002f130 service glance] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:32:03.157212 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 990/1967] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Jun 15 10:32:03 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:32:09.088207 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ca0d99-240c-4cbb-ac8f-5d14753ad424 None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:32:09.089708 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-78ca0d99-240c-4cbb-ac8f-5d14753ad424 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:32:09.155763 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ca0d99-240c-4cbb-ac8f-5d14753ad424 None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:32:09.159580 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aaf6f87-b456-4523-be83-b29111baeb62 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20d1327553404b80aecee87c78fcb52b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '559b3e713f5844b6ae37151d56dd419f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:32:09.160237 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aaf6f87-b456-4523-be83-b29111baeb62 service cinder] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:32:09.160585 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aaf6f87-b456-4523-be83-b29111baeb62 service cinder] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:32:09.160871 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aaf6f87-b456-4523-be83-b29111baeb62 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:32:09.162697 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-2aaf6f87-b456-4523-be83-b29111baeb62 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:32:09.228731 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aaf6f87-b456-4523-be83-b29111baeb62 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:32:09.230160 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aaf6f87-b456-4523-be83-b29111baeb62 service cinder] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:32:09.239783 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 978/1968] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Jun 15 10:32:09 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:32:10.165746 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebf76e0-3c27-4995-85ca-b5b45002f130 service glance] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:32:10.170366 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-7ebf76e0-3c27-4995-85ca-b5b45002f130 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:32:10.262056 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:32:10.271801 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebf76e0-3c27-4995-85ca-b5b45002f130 service glance] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:32:10.279002 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8566c39-c33e-4e27-848e-913d8217a6e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:32:10.280243 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8566c39-c33e-4e27-848e-913d8217a6e3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:32:10.280734 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8566c39-c33e-4e27-848e-913d8217a6e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:32:10.281437 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8566c39-c33e-4e27-848e-913d8217a6e3 admin admin] PATH_INFO: `/v3/users/885ecc9a6f9a476eb38a8c33f21e0d61` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:32:10.283693 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8566c39-c33e-4e27-848e-913d8217a6e3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=885ecc9a6f9a476eb38a8c33f21e0d61)` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:32:10.286405 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8566c39-c33e-4e27-848e-913d8217a6e3 admin admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:32:10.365047 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-d8566c39-c33e-4e27-848e-913d8217a6e3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '885ecc9a6f9a476eb38a8c33f21e0d61'} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:32:10.372898 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-d8566c39-c33e-4e27-848e-913d8217a6e3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '885ecc9a6f9a476eb38a8c33f21e0d61'} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:32:10.373717 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-d8566c39-c33e-4e27-848e-913d8217a6e3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '885ecc9a6f9a476eb38a8c33f21e0d61'} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:32:10.374391 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-d8566c39-c33e-4e27-848e-913d8217a6e3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '885ecc9a6f9a476eb38a8c33f21e0d61'} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:32:10.383232 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-d8566c39-c33e-4e27-848e-913d8217a6e3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '885ecc9a6f9a476eb38a8c33f21e0d61'} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:32:10.404135 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 991/1969] 10.222.0.49 () {58 vars in 1223 bytes} [Wed Jun 15 10:32:10 2022] DELETE /identity/v3/users/885ecc9a6f9a476eb38a8c33f21e0d61 => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:32:10.408768 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7ad6a1-9ef2-44b9-804e-79ec3baad38b None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:32:10.409611 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7ad6a1-9ef2-44b9-804e-79ec3baad38b None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:32:10.410097 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7ad6a1-9ef2-44b9-804e-79ec3baad38b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:32:10.428336 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-5c7ad6a1-9ef2-44b9-804e-79ec3baad38b None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:32:10.429942 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-5c7ad6a1-9ef2-44b9-804e-79ec3baad38b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:32:10.478686 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:32:10.517968 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 979/1970] 10.222.0.49 () {58 vars in 957 bytes} [Wed Jun 15 10:32:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:32:10.531471 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8566c39-c33e-4e27-848e-913d8217a6e3 admin admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:32:10.533527 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-d8566c39-c33e-4e27-848e-913d8217a6e3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:32:10.605424 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8566c39-c33e-4e27-848e-913d8217a6e3 admin admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:32:10.610307 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4c0cc7-5353-4d98-8cef-90cd7b7a2a9b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0cb44a17d205431793e9a8791e477a32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '559b3e713f5844b6ae37151d56dd419f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:32:10.611124 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4c0cc7-5353-4d98-8cef-90cd7b7a2a9b service neutron] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:32:10.611508 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4c0cc7-5353-4d98-8cef-90cd7b7a2a9b service neutron] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:32:10.611840 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4c0cc7-5353-4d98-8cef-90cd7b7a2a9b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:32:10.624479 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4c0cc7-5353-4d98-8cef-90cd7b7a2a9b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:32:10.625937 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4c0cc7-5353-4d98-8cef-90cd7b7a2a9b service neutron] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:32:10.638476 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 992/1971] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Jun 15 10:32:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:32:11.056634 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7ad6a1-9ef2-44b9-804e-79ec3baad38b None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:32:11.058622 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-5c7ad6a1-9ef2-44b9-804e-79ec3baad38b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:32:11.102746 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7ad6a1-9ef2-44b9-804e-79ec3baad38b None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:32:11.108081 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3986edd9-8618-4e72-b80b-32f6305a9ee6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:32:11.108985 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3986edd9-8618-4e72-b80b-32f6305a9ee6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:32:11.109277 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3986edd9-8618-4e72-b80b-32f6305a9ee6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:32:11.109565 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3986edd9-8618-4e72-b80b-32f6305a9ee6 admin admin] PATH_INFO: `/v3/projects/ca2186cc0fd842378a145da171b48417` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:32:11.110798 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3986edd9-8618-4e72-b80b-32f6305a9ee6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ca2186cc0fd842378a145da171b48417)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:32:11.112762 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3986edd9-8618-4e72-b80b-32f6305a9ee6 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:32:11.162640 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-3986edd9-8618-4e72-b80b-32f6305a9ee6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ca2186cc0fd842378a145da171b48417', 'request_id': 'req-3986edd9-8618-4e72-b80b-32f6305a9ee6'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:32:11.174503 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-3986edd9-8618-4e72-b80b-32f6305a9ee6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ca2186cc0fd842378a145da171b48417', 'request_id': 'req-3986edd9-8618-4e72-b80b-32f6305a9ee6'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:32:11.180761 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-3986edd9-8618-4e72-b80b-32f6305a9ee6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ca2186cc0fd842378a145da171b48417', 'request_id': 'req-3986edd9-8618-4e72-b80b-32f6305a9ee6'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:32:11.192617 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-3986edd9-8618-4e72-b80b-32f6305a9ee6 admin admin] The token cache is being invalidate because project ca2186cc0fd842378a145da171b48417 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80120) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 10:32:11.192913 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-3986edd9-8618-4e72-b80b-32f6305a9ee6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:32:11.193860 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-3986edd9-8618-4e72-b80b-32f6305a9ee6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:32:11.196132 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 980/1972] 10.222.0.49 () {58 vars in 1232 bytes} [Wed Jun 15 10:32:11 2022] DELETE /identity/v3/projects/ca2186cc0fd842378a145da171b48417 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:32:11.201534 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e334aa7b-34f4-4e38-ab2d-c97be3fed93d None None] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:32:11.202130 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e334aa7b-34f4-4e38-ab2d-c97be3fed93d None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:32:11.202130 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e334aa7b-34f4-4e38-ab2d-c97be3fed93d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:32:11.223703 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.auth.core [None req-e334aa7b-34f4-4e38-ab2d-c97be3fed93d None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:32:11.225007 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-e334aa7b-34f4-4e38-ab2d-c97be3fed93d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:32:11.291148 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 993/1973] 10.222.0.49 () {58 vars in 957 bytes} [Wed Jun 15 10:32:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:32:11.299204 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e4db0e-9a9a-416a-be00-17fe0b5dcfaf None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:32:11.299501 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e4db0e-9a9a-416a-be00-17fe0b5dcfaf None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:32:11.299783 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e4db0e-9a9a-416a-be00-17fe0b5dcfaf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:32:11.321828 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-57e4db0e-9a9a-416a-be00-17fe0b5dcfaf None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:32:11.323522 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-57e4db0e-9a9a-416a-be00-17fe0b5dcfaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:32:11.341464 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 981/1974] 10.222.0.49 () {58 vars in 957 bytes} [Wed Jun 15 10:32:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 10:32:11.345617 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e334aa7b-34f4-4e38-ab2d-c97be3fed93d None None] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:32:11.356830 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e334aa7b-34f4-4e38-ab2d-c97be3fed93d None None] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:32:11.366633 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19bff12a-42f5-4bad-a1a1-00a58e08651a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:32:11.367322 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bff12a-42f5-4bad-a1a1-00a58e08651a admin admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:32:11.367596 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bff12a-42f5-4bad-a1a1-00a58e08651a admin admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:32:11.367870 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bff12a-42f5-4bad-a1a1-00a58e08651a admin admin] PATH_INFO: `/v3/domains` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:32:11.368596 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19bff12a-42f5-4bad-a1a1-00a58e08651a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:32:11.368901 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19bff12a-42f5-4bad-a1a1-00a58e08651a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:32:11.370501 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19bff12a-42f5-4bad-a1a1-00a58e08651a admin admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:32:11.387201 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 994/1975] 10.222.0.49 () {58 vars in 1152 bytes} [Wed Jun 15 10:32:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:32:11.389601 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e4db0e-9a9a-416a-be00-17fe0b5dcfaf None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:32:11.407660 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e4db0e-9a9a-416a-be00-17fe0b5dcfaf None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:32:11.415693 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6fe710-e431-41ca-a075-cab27750d910 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:32:11.417410 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6fe710-e431-41ca-a075-cab27750d910 admin admin] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:32:11.417875 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6fe710-e431-41ca-a075-cab27750d910 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:32:11.418538 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6fe710-e431-41ca-a075-cab27750d910 admin admin] PATH_INFO: `/v3/projects` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:32:11.420287 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6fe710-e431-41ca-a075-cab27750d910 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:32:11.422650 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6fe710-e431-41ca-a075-cab27750d910 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:32:11.458398 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 982/1976] 10.222.0.49 () {60 vars in 1152 bytes} [Wed Jun 15 10:32:11 2022] POST /identity/v3/projects => generated 377 bytes in 70 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 10:32:11.463628 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19bff12a-42f5-4bad-a1a1-00a58e08651a admin admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:32:11.480704 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19bff12a-42f5-4bad-a1a1-00a58e08651a admin admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:32:11.485410 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:32:11.579912 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d321ffa-50b8-4d3e-a93b-8923fa33fe83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:32:11.580946 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d321ffa-50b8-4d3e-a93b-8923fa33fe83 admin admin] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:32:11.581502 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d321ffa-50b8-4d3e-a93b-8923fa33fe83 admin admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:32:11.581789 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d321ffa-50b8-4d3e-a93b-8923fa33fe83 admin admin] PATH_INFO: `/v3/users` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:32:11.582637 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d321ffa-50b8-4d3e-a93b-8923fa33fe83 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:32:11.584636 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d321ffa-50b8-4d3e-a93b-8923fa33fe83 admin admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:32:11.618795 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 995/1977] 10.222.0.49 () {60 vars in 1143 bytes} [Wed Jun 15 10:32:11 2022] POST /identity/v3/users => generated 289 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 10:32:11.620416 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6fe710-e431-41ca-a075-cab27750d910 admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:32:11.635327 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6fe710-e431-41ca-a075-cab27750d910 admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:32:11.642900 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c8b1ef-3eda-498d-83a7-b95ac93932b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:32:11.643785 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c8b1ef-3eda-498d-83a7-b95ac93932b5 admin admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:32:11.644269 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c8b1ef-3eda-498d-83a7-b95ac93932b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:32:11.644713 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c8b1ef-3eda-498d-83a7-b95ac93932b5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:32:11.645414 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c8b1ef-3eda-498d-83a7-b95ac93932b5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:32:11.647039 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c8b1ef-3eda-498d-83a7-b95ac93932b5 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:32:11.671365 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d321ffa-50b8-4d3e-a93b-8923fa33fe83 admin admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:32:11.671781 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 983/1978] 10.222.0.49 () {58 vars in 1121 bytes} [Wed Jun 15 10:32:11 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 10:32:11.685048 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d321ffa-50b8-4d3e-a93b-8923fa33fe83 admin admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:32:11.719256 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e80280-2b56-4dff-a9c5-4695586038a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:32:11.720553 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e80280-2b56-4dff-a9c5-4695586038a3 admin admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:32:11.720830 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e80280-2b56-4dff-a9c5-4695586038a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:32:11.721112 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e80280-2b56-4dff-a9c5-4695586038a3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:32:11.722697 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e80280-2b56-4dff-a9c5-4695586038a3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:32:11.724832 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e80280-2b56-4dff-a9c5-4695586038a3 admin admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:32:11.748017 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c8b1ef-3eda-498d-83a7-b95ac93932b5 admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:32:11.757148 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 996/1979] 10.222.0.49 () {58 vars in 1121 bytes} [Wed Jun 15 10:32:11 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 10:32:11.763941 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c8b1ef-3eda-498d-83a7-b95ac93932b5 admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:32:11.771368 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5121e194-bda5-497b-9ebb-3548c9e1ad48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:32:11.772352 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5121e194-bda5-497b-9ebb-3548c9e1ad48 admin admin] REQUEST_METHOD: `PUT` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:32:11.772645 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5121e194-bda5-497b-9ebb-3548c9e1ad48 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:32:11.772971 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5121e194-bda5-497b-9ebb-3548c9e1ad48 admin admin] PATH_INFO: `/v3/projects/3f66feeddeba414296d6f0a04e97457a/users/de3df1675add45558d3fd4183dd1785e/roles/6876e57cc51c448097ab109d35b2a3da` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:32:11.790351 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5121e194-bda5-497b-9ebb-3548c9e1ad48 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3f66feeddeba414296d6f0a04e97457a, user_id=de3df1675add45558d3fd4183dd1785e, role_id=6876e57cc51c448097ab109d35b2a3da)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:32:11.793152 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5121e194-bda5-497b-9ebb-3548c9e1ad48 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:32:11.824587 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 984/1980] 10.222.0.49 () {60 vars in 1482 bytes} [Wed Jun 15 10:32:11 2022] PUT /identity/v3/projects/3f66feeddeba414296d6f0a04e97457a/users/de3df1675add45558d3fd4183dd1785e/roles/6876e57cc51c448097ab109d35b2a3da => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:32:11.835257 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cbcd04-7095-4555-a8c7-04d0ef2393ef None None] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:32:11.836338 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cbcd04-7095-4555-a8c7-04d0ef2393ef None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:32:11.836636 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cbcd04-7095-4555-a8c7-04d0ef2393ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:32:11.876679 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.auth.core [None req-75cbcd04-7095-4555-a8c7-04d0ef2393ef None None] MFA Rules not processed for user `de3df1675add45558d3fd4183dd1785e`. Rule list: `[]` (Enabled: `True`). {{(pid=80119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:32:11.878002 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-75cbcd04-7095-4555-a8c7-04d0ef2393ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:32:11.989249 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5121e194-bda5-497b-9ebb-3548c9e1ad48 admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:32:11.991654 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-5121e194-bda5-497b-9ebb-3548c9e1ad48 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:32:12.033986 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 997/1981] 10.222.0.49 () {58 vars in 957 bytes} [Wed Jun 15 10:32:11 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:32:12.061897 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75cbcd04-7095-4555-a8c7-04d0ef2393ef None None] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:32:12.063725 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-75cbcd04-7095-4555-a8c7-04d0ef2393ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:32:12.124235 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:32:12.133513 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5121e194-bda5-497b-9ebb-3548c9e1ad48 admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:32:12.152683 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f51b6db-9147-43dc-8ad0-243d12b1eecd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:32:12.153412 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f51b6db-9147-43dc-8ad0-243d12b1eecd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:32:12.153706 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f51b6db-9147-43dc-8ad0-243d12b1eecd admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:32:12.153988 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f51b6db-9147-43dc-8ad0-243d12b1eecd admin admin] PATH_INFO: `/v3/users/19dbc90156e74a50aa55d9c9f13bbe4d` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:32:12.157006 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f51b6db-9147-43dc-8ad0-243d12b1eecd admin admin] RBAC: Authorizing `identity:delete_user(user_id=19dbc90156e74a50aa55d9c9f13bbe4d)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:32:12.159275 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f51b6db-9147-43dc-8ad0-243d12b1eecd admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:32:12.191040 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75cbcd04-7095-4555-a8c7-04d0ef2393ef None None] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:32:12.197932 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9036df-ec40-4eee-8b2e-30cf646eb00d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20d1327553404b80aecee87c78fcb52b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '559b3e713f5844b6ae37151d56dd419f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:32:12.198797 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9036df-ec40-4eee-8b2e-30cf646eb00d service cinder] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:32:12.199564 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9036df-ec40-4eee-8b2e-30cf646eb00d service cinder] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:32:12.199837 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9036df-ec40-4eee-8b2e-30cf646eb00d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:32:12.217038 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a9036df-ec40-4eee-8b2e-30cf646eb00d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:32:12.220115 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a9036df-ec40-4eee-8b2e-30cf646eb00d service cinder] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:32:12.237167 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 998/1982] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Jun 15 10:32:12 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:32:12.261892 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-3f51b6db-9147-43dc-8ad0-243d12b1eecd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '19dbc90156e74a50aa55d9c9f13bbe4d'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:32:12.271345 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-3f51b6db-9147-43dc-8ad0-243d12b1eecd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '19dbc90156e74a50aa55d9c9f13bbe4d'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:32:12.272246 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-3f51b6db-9147-43dc-8ad0-243d12b1eecd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '19dbc90156e74a50aa55d9c9f13bbe4d'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:32:12.273555 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-3f51b6db-9147-43dc-8ad0-243d12b1eecd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '19dbc90156e74a50aa55d9c9f13bbe4d'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:32:12.286860 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-3f51b6db-9147-43dc-8ad0-243d12b1eecd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '19dbc90156e74a50aa55d9c9f13bbe4d'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:32:12.341604 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 985/1983] 10.222.0.49 () {58 vars in 1223 bytes} [Wed Jun 15 10:32:11 2022] DELETE /identity/v3/users/19dbc90156e74a50aa55d9c9f13bbe4d => generated 0 bytes in 353 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:32:12.346180 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db775dae-1817-4b13-818b-0e48dfefea58 None None] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:32:12.347162 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db775dae-1817-4b13-818b-0e48dfefea58 None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:32:12.347526 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db775dae-1817-4b13-818b-0e48dfefea58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:32:12.366631 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.auth.core [None req-db775dae-1817-4b13-818b-0e48dfefea58 None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:32:12.368593 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-db775dae-1817-4b13-818b-0e48dfefea58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:32:12.384544 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:32:12.442429 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 999/1984] 10.222.0.49 () {58 vars in 957 bytes} [Wed Jun 15 10:32:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:32:12.452835 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f51b6db-9147-43dc-8ad0-243d12b1eecd admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:32:12.456080 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-3f51b6db-9147-43dc-8ad0-243d12b1eecd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:32:12.573908 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f51b6db-9147-43dc-8ad0-243d12b1eecd admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:32:12.580374 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8472c17f-3c5a-426e-9b14-e0448e43ff85 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0cb44a17d205431793e9a8791e477a32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '559b3e713f5844b6ae37151d56dd419f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:32:12.581871 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8472c17f-3c5a-426e-9b14-e0448e43ff85 service neutron] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:32:12.582306 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8472c17f-3c5a-426e-9b14-e0448e43ff85 service neutron] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:32:12.582729 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8472c17f-3c5a-426e-9b14-e0448e43ff85 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:32:12.639110 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8472c17f-3c5a-426e-9b14-e0448e43ff85 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:32:12.654060 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8472c17f-3c5a-426e-9b14-e0448e43ff85 service neutron] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:32:12.678893 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 986/1985] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Jun 15 10:32:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 227 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:32:13.013166 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db775dae-1817-4b13-818b-0e48dfefea58 None None] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:32:13.014565 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-db775dae-1817-4b13-818b-0e48dfefea58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:32:13.028694 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db775dae-1817-4b13-818b-0e48dfefea58 None None] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:32:13.033702 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5026bf7e-3fc7-44a3-95de-70c7abea2bb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:32:13.034573 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5026bf7e-3fc7-44a3-95de-70c7abea2bb3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:32:13.034855 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5026bf7e-3fc7-44a3-95de-70c7abea2bb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:32:13.035148 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5026bf7e-3fc7-44a3-95de-70c7abea2bb3 admin admin] PATH_INFO: `/v3/projects/3294948606364d24bc5beb0d13eb51b0` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:32:13.036477 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5026bf7e-3fc7-44a3-95de-70c7abea2bb3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3294948606364d24bc5beb0d13eb51b0)` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:32:13.039044 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5026bf7e-3fc7-44a3-95de-70c7abea2bb3 admin admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:32:13.094464 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-5026bf7e-3fc7-44a3-95de-70c7abea2bb3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3294948606364d24bc5beb0d13eb51b0', 'request_id': 'req-5026bf7e-3fc7-44a3-95de-70c7abea2bb3'} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:32:13.105578 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-5026bf7e-3fc7-44a3-95de-70c7abea2bb3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3294948606364d24bc5beb0d13eb51b0', 'request_id': 'req-5026bf7e-3fc7-44a3-95de-70c7abea2bb3'} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:32:13.111858 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-5026bf7e-3fc7-44a3-95de-70c7abea2bb3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3294948606364d24bc5beb0d13eb51b0', 'request_id': 'req-5026bf7e-3fc7-44a3-95de-70c7abea2bb3'} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:32:13.124029 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-5026bf7e-3fc7-44a3-95de-70c7abea2bb3 admin admin] The token cache is being invalidate because project 3294948606364d24bc5beb0d13eb51b0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80119) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 10:32:13.124334 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-5026bf7e-3fc7-44a3-95de-70c7abea2bb3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:32:13.125046 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-5026bf7e-3fc7-44a3-95de-70c7abea2bb3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:32:13.126700 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1000/1986] 10.222.0.49 () {58 vars in 1232 bytes} [Wed Jun 15 10:32:13 2022] DELETE /identity/v3/projects/3294948606364d24bc5beb0d13eb51b0 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:32:13.134547 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8472c17f-3c5a-426e-9b14-e0448e43ff85 service neutron] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:32:13.145590 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8472c17f-3c5a-426e-9b14-e0448e43ff85 service neutron] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:32:13.245679 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ada5ea-ca15-4021-9d53-60e6b4326e4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:32:13.246711 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ada5ea-ca15-4021-9d53-60e6b4326e4a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:32:13.247006 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ada5ea-ca15-4021-9d53-60e6b4326e4a admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:32:13.247529 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ada5ea-ca15-4021-9d53-60e6b4326e4a admin admin] PATH_INFO: `/v3/users/91b75c5f75f24de08481f4fcefdbac7b` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:32:13.248874 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ada5ea-ca15-4021-9d53-60e6b4326e4a admin admin] RBAC: Authorizing `identity:delete_user(user_id=91b75c5f75f24de08481f4fcefdbac7b)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:32:13.251038 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ada5ea-ca15-4021-9d53-60e6b4326e4a admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:32:13.275652 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:32:13.307671 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-d2ada5ea-ca15-4021-9d53-60e6b4326e4a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '91b75c5f75f24de08481f4fcefdbac7b'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:32:13.316694 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-d2ada5ea-ca15-4021-9d53-60e6b4326e4a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '91b75c5f75f24de08481f4fcefdbac7b'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:32:13.317401 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-d2ada5ea-ca15-4021-9d53-60e6b4326e4a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '91b75c5f75f24de08481f4fcefdbac7b'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:32:13.318014 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-d2ada5ea-ca15-4021-9d53-60e6b4326e4a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '91b75c5f75f24de08481f4fcefdbac7b'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:32:13.327662 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-d2ada5ea-ca15-4021-9d53-60e6b4326e4a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '91b75c5f75f24de08481f4fcefdbac7b'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:32:13.354843 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 987/1987] 10.222.0.49 () {58 vars in 1223 bytes} [Wed Jun 15 10:32:13 2022] DELETE /identity/v3/users/91b75c5f75f24de08481f4fcefdbac7b => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:32:13.838543 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5026bf7e-3fc7-44a3-95de-70c7abea2bb3 admin admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:32:13.840399 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-5026bf7e-3fc7-44a3-95de-70c7abea2bb3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:32:13.853239 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5026bf7e-3fc7-44a3-95de-70c7abea2bb3 admin admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:32:13.925056 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8961a2e6-7d76-4cff-803b-c597009749b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:32:13.926183 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8961a2e6-7d76-4cff-803b-c597009749b6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:32:13.926586 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8961a2e6-7d76-4cff-803b-c597009749b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:32:13.927412 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8961a2e6-7d76-4cff-803b-c597009749b6 admin admin] PATH_INFO: `/v3/projects/bc59f64c4b7649ac9ba1934c64e5ce60` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:32:13.928494 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8961a2e6-7d76-4cff-803b-c597009749b6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bc59f64c4b7649ac9ba1934c64e5ce60)` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:32:13.931936 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8961a2e6-7d76-4cff-803b-c597009749b6 admin admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:32:13.986903 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:32:13.992193 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-8961a2e6-7d76-4cff-803b-c597009749b6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bc59f64c4b7649ac9ba1934c64e5ce60', 'request_id': 'req-8961a2e6-7d76-4cff-803b-c597009749b6'} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:32:14.001009 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-8961a2e6-7d76-4cff-803b-c597009749b6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bc59f64c4b7649ac9ba1934c64e5ce60', 'request_id': 'req-8961a2e6-7d76-4cff-803b-c597009749b6'} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:32:14.006576 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-8961a2e6-7d76-4cff-803b-c597009749b6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bc59f64c4b7649ac9ba1934c64e5ce60', 'request_id': 'req-8961a2e6-7d76-4cff-803b-c597009749b6'} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:32:14.018187 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-8961a2e6-7d76-4cff-803b-c597009749b6 admin admin] The token cache is being invalidate because project bc59f64c4b7649ac9ba1934c64e5ce60 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80119) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 10:32:14.018886 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-8961a2e6-7d76-4cff-803b-c597009749b6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:32:14.019574 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-8961a2e6-7d76-4cff-803b-c597009749b6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:32:14.021221 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1001/1988] 10.222.0.49 () {58 vars in 1232 bytes} [Wed Jun 15 10:32:13 2022] DELETE /identity/v3/projects/bc59f64c4b7649ac9ba1934c64e5ce60 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:32:14.025148 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ada5ea-ca15-4021-9d53-60e6b4326e4a admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:32:14.026595 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-d2ada5ea-ca15-4021-9d53-60e6b4326e4a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:32:14.039570 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ada5ea-ca15-4021-9d53-60e6b4326e4a admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:32:14.094191 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67fe00be-4ed2-400f-836d-0b49e0d630fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:32:14.095071 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fe00be-4ed2-400f-836d-0b49e0d630fa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:32:14.095465 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fe00be-4ed2-400f-836d-0b49e0d630fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:32:14.095677 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fe00be-4ed2-400f-836d-0b49e0d630fa admin admin] PATH_INFO: `/v3/users/cf66f62d7d664e1d8e4035a465934fd4` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:32:14.096864 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67fe00be-4ed2-400f-836d-0b49e0d630fa admin admin] RBAC: Authorizing `identity:delete_user(user_id=cf66f62d7d664e1d8e4035a465934fd4)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:32:14.098768 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67fe00be-4ed2-400f-836d-0b49e0d630fa admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:32:14.145838 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-67fe00be-4ed2-400f-836d-0b49e0d630fa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cf66f62d7d664e1d8e4035a465934fd4'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:32:14.152557 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-67fe00be-4ed2-400f-836d-0b49e0d630fa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cf66f62d7d664e1d8e4035a465934fd4'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:32:14.153221 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-67fe00be-4ed2-400f-836d-0b49e0d630fa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cf66f62d7d664e1d8e4035a465934fd4'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:32:14.153822 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-67fe00be-4ed2-400f-836d-0b49e0d630fa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cf66f62d7d664e1d8e4035a465934fd4'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:32:14.161399 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-67fe00be-4ed2-400f-836d-0b49e0d630fa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cf66f62d7d664e1d8e4035a465934fd4'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:32:14.180113 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 988/1989] 10.222.0.49 () {58 vars in 1223 bytes} [Wed Jun 15 10:32:14 2022] DELETE /identity/v3/users/cf66f62d7d664e1d8e4035a465934fd4 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:32:14.471850 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8961a2e6-7d76-4cff-803b-c597009749b6 admin admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:32:14.474221 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-8961a2e6-7d76-4cff-803b-c597009749b6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:32:14.493578 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8961a2e6-7d76-4cff-803b-c597009749b6 admin admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:32:14.583398 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4795a8c-e0ae-43d4-98ff-036d7e7d48f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:32:14.584836 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4795a8c-e0ae-43d4-98ff-036d7e7d48f6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:32:14.587284 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4795a8c-e0ae-43d4-98ff-036d7e7d48f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:32:14.587890 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4795a8c-e0ae-43d4-98ff-036d7e7d48f6 admin admin] PATH_INFO: `/v3/projects/965147f94aa84a8ca784939a9d5aafbc` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:32:14.589514 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4795a8c-e0ae-43d4-98ff-036d7e7d48f6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=965147f94aa84a8ca784939a9d5aafbc)` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:32:14.591732 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4795a8c-e0ae-43d4-98ff-036d7e7d48f6 admin admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:32:14.690369 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-a4795a8c-e0ae-43d4-98ff-036d7e7d48f6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '965147f94aa84a8ca784939a9d5aafbc', 'request_id': 'req-a4795a8c-e0ae-43d4-98ff-036d7e7d48f6'} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:32:14.703619 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-a4795a8c-e0ae-43d4-98ff-036d7e7d48f6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '965147f94aa84a8ca784939a9d5aafbc', 'request_id': 'req-a4795a8c-e0ae-43d4-98ff-036d7e7d48f6'} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:32:14.715879 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-a4795a8c-e0ae-43d4-98ff-036d7e7d48f6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '965147f94aa84a8ca784939a9d5aafbc', 'request_id': 'req-a4795a8c-e0ae-43d4-98ff-036d7e7d48f6'} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:32:14.728509 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-a4795a8c-e0ae-43d4-98ff-036d7e7d48f6 admin admin] The token cache is being invalidate because project 965147f94aa84a8ca784939a9d5aafbc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80119) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 10:32:14.728878 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-a4795a8c-e0ae-43d4-98ff-036d7e7d48f6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:32:14.729587 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-a4795a8c-e0ae-43d4-98ff-036d7e7d48f6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:32:14.731273 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1002/1990] 10.222.0.49 () {58 vars in 1232 bytes} [Wed Jun 15 10:32:14 2022] DELETE /identity/v3/projects/965147f94aa84a8ca784939a9d5aafbc => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:32:14.737052 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5deb1c-8f78-4edc-b42b-e92a9c77d705 None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:32:14.737341 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5deb1c-8f78-4edc-b42b-e92a9c77d705 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:32:14.737615 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5deb1c-8f78-4edc-b42b-e92a9c77d705 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:32:14.755732 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-9f5deb1c-8f78-4edc-b42b-e92a9c77d705 None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:32:14.756947 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-9f5deb1c-8f78-4edc-b42b-e92a9c77d705 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:32:14.769625 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:DELETE=17,keystone:SELECT=81,keystone:INSERT=2 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:32:14.827633 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 989/1991] 10.222.0.49 () {58 vars in 957 bytes} [Wed Jun 15 10:32:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:32:14.835852 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c196480-25a2-4ba4-a1d5-bbec5b9494cd None None] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:32:14.836147 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c196480-25a2-4ba4-a1d5-bbec5b9494cd None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:32:14.836428 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c196480-25a2-4ba4-a1d5-bbec5b9494cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:32:14.855619 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.auth.core [None req-1c196480-25a2-4ba4-a1d5-bbec5b9494cd None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:32:14.856824 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-1c196480-25a2-4ba4-a1d5-bbec5b9494cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:32:14.864943 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1003/1992] 10.222.0.49 () {58 vars in 957 bytes} [Wed Jun 15 10:32:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 10:32:14.869463 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5deb1c-8f78-4edc-b42b-e92a9c77d705 None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:32:14.879081 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5deb1c-8f78-4edc-b42b-e92a9c77d705 None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:32:14.884332 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf35f5c-d97f-4220-8c62-4af53bfcd95b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:32:14.885002 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf35f5c-d97f-4220-8c62-4af53bfcd95b admin admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:32:14.885285 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf35f5c-d97f-4220-8c62-4af53bfcd95b admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:32:14.885561 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf35f5c-d97f-4220-8c62-4af53bfcd95b admin admin] PATH_INFO: `/v3/domains` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:32:14.886410 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf35f5c-d97f-4220-8c62-4af53bfcd95b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:32:14.886725 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf35f5c-d97f-4220-8c62-4af53bfcd95b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:32:14.888366 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf35f5c-d97f-4220-8c62-4af53bfcd95b admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:32:14.900232 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 990/1993] 10.222.0.49 () {58 vars in 1152 bytes} [Wed Jun 15 10:32:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:32:14.904526 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c196480-25a2-4ba4-a1d5-bbec5b9494cd None None] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:32:14.914156 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c196480-25a2-4ba4-a1d5-bbec5b9494cd None None] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:32:14.918943 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47419aec-5bc4-4450-95ce-1fb6cf029374 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:32:14.919585 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47419aec-5bc4-4450-95ce-1fb6cf029374 admin admin] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:32:14.919875 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47419aec-5bc4-4450-95ce-1fb6cf029374 admin admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:32:14.920156 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47419aec-5bc4-4450-95ce-1fb6cf029374 admin admin] PATH_INFO: `/v3/projects` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:32:14.920839 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47419aec-5bc4-4450-95ce-1fb6cf029374 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:32:14.922744 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47419aec-5bc4-4450-95ce-1fb6cf029374 admin admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:32:14.953317 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1004/1994] 10.222.0.49 () {60 vars in 1152 bytes} [Wed Jun 15 10:32:14 2022] POST /identity/v3/projects => generated 373 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 10:32:14.954510 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf35f5c-d97f-4220-8c62-4af53bfcd95b admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:32:14.964498 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf35f5c-d97f-4220-8c62-4af53bfcd95b admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:32:15.021873 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffdf2da-628b-440a-b39e-78a27ef87470 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:32:15.022555 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffdf2da-628b-440a-b39e-78a27ef87470 admin admin] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:32:15.022846 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffdf2da-628b-440a-b39e-78a27ef87470 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:32:15.023195 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffdf2da-628b-440a-b39e-78a27ef87470 admin admin] PATH_INFO: `/v3/users` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:32:15.023922 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dffdf2da-628b-440a-b39e-78a27ef87470 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:32:15.025929 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dffdf2da-628b-440a-b39e-78a27ef87470 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:32:15.045999 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 991/1995] 10.222.0.49 () {60 vars in 1143 bytes} [Wed Jun 15 10:32:14 2022] POST /identity/v3/users => generated 287 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 10:32:15.050206 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47419aec-5bc4-4450-95ce-1fb6cf029374 admin admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:32:15.061038 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47419aec-5bc4-4450-95ce-1fb6cf029374 admin admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:32:15.065615 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca68149-c91b-4da5-a275-68f09975c91f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:32:15.066272 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca68149-c91b-4da5-a275-68f09975c91f admin admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:32:15.066553 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca68149-c91b-4da5-a275-68f09975c91f admin admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:32:15.066826 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca68149-c91b-4da5-a275-68f09975c91f admin admin] PATH_INFO: `/v3/roles` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:32:15.067586 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca68149-c91b-4da5-a275-68f09975c91f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:32:15.069196 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca68149-c91b-4da5-a275-68f09975c91f admin admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:32:15.079222 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1005/1996] 10.222.0.49 () {58 vars in 1121 bytes} [Wed Jun 15 10:32:15 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 10:32:15.082985 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffdf2da-628b-440a-b39e-78a27ef87470 admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:32:15.092864 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffdf2da-628b-440a-b39e-78a27ef87470 admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:32:15.097457 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b082ce9-25c5-4bf3-b003-115af319313f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:32:15.098111 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b082ce9-25c5-4bf3-b003-115af319313f admin admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:32:15.098398 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b082ce9-25c5-4bf3-b003-115af319313f admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:32:15.098677 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b082ce9-25c5-4bf3-b003-115af319313f admin admin] PATH_INFO: `/v3/roles` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:32:15.099445 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b082ce9-25c5-4bf3-b003-115af319313f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:32:15.101533 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b082ce9-25c5-4bf3-b003-115af319313f admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:32:15.111382 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 992/1997] 10.222.0.49 () {58 vars in 1121 bytes} [Wed Jun 15 10:32:15 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 10:32:15.115375 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca68149-c91b-4da5-a275-68f09975c91f admin admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:32:15.125664 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca68149-c91b-4da5-a275-68f09975c91f admin admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:32:15.130211 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053351bb-3438-44df-89db-368203d23fb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:32:15.131077 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053351bb-3438-44df-89db-368203d23fb7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:32:15.131388 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053351bb-3438-44df-89db-368203d23fb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:32:15.131656 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053351bb-3438-44df-89db-368203d23fb7 admin admin] PATH_INFO: `/v3/projects/75f96c492c964ea197288dc60d5d60a7/users/40fd04a3958041d398b14cf5e515214c/roles/6876e57cc51c448097ab109d35b2a3da` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:32:15.145575 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-053351bb-3438-44df-89db-368203d23fb7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=75f96c492c964ea197288dc60d5d60a7, user_id=40fd04a3958041d398b14cf5e515214c, role_id=6876e57cc51c448097ab109d35b2a3da)` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:32:15.148284 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-053351bb-3438-44df-89db-368203d23fb7 admin admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:32:15.155935 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=85,keystone:DELETE=11,keystone:INSERT=4 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:32:15.162432 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1006/1998] 10.222.0.49 () {60 vars in 1482 bytes} [Wed Jun 15 10:32:15 2022] PUT /identity/v3/projects/75f96c492c964ea197288dc60d5d60a7/users/40fd04a3958041d398b14cf5e515214c/roles/6876e57cc51c448097ab109d35b2a3da => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:32:15.168000 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17930815-8f9e-468a-a0b5-bde7c21f712b None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:32:15.168295 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17930815-8f9e-468a-a0b5-bde7c21f712b None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:32:15.168578 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17930815-8f9e-468a-a0b5-bde7c21f712b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:32:15.188116 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-17930815-8f9e-468a-a0b5-bde7c21f712b None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:32:15.189250 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-17930815-8f9e-468a-a0b5-bde7c21f712b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:32:15.252436 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 993/1999] 10.222.0.49 () {58 vars in 957 bytes} [Wed Jun 15 10:32:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:32:15.261858 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053351bb-3438-44df-89db-368203d23fb7 admin admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:32:15.263608 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-053351bb-3438-44df-89db-368203d23fb7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:32:15.319309 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053351bb-3438-44df-89db-368203d23fb7 admin admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:32:15.323109 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6937ce24-1a1a-48d5-ba9b-ffaf8c5c6398 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0cb44a17d205431793e9a8791e477a32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '559b3e713f5844b6ae37151d56dd419f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:32:15.323831 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6937ce24-1a1a-48d5-ba9b-ffaf8c5c6398 service neutron] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:32:15.324116 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6937ce24-1a1a-48d5-ba9b-ffaf8c5c6398 service neutron] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:32:15.324397 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6937ce24-1a1a-48d5-ba9b-ffaf8c5c6398 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:32:15.333346 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6937ce24-1a1a-48d5-ba9b-ffaf8c5c6398 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:32:15.334813 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6937ce24-1a1a-48d5-ba9b-ffaf8c5c6398 service neutron] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:32:15.344663 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1007/2000] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Jun 15 10:32:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:32:16.026277 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd9cc44-0d57-4686-a8a8-83b943b44df4 None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:32:16.027303 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd9cc44-0d57-4686-a8a8-83b943b44df4 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:32:16.027724 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd9cc44-0d57-4686-a8a8-83b943b44df4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:32:16.060563 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-8dd9cc44-0d57-4686-a8a8-83b943b44df4 None None] MFA Rules not processed for user `40fd04a3958041d398b14cf5e515214c`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:32:16.062183 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-8dd9cc44-0d57-4686-a8a8-83b943b44df4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:32:16.077303 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:32:16.138876 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 994/2001] 10.222.0.49 () {58 vars in 957 bytes} [Wed Jun 15 10:32:16 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:32:16.143273 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6937ce24-1a1a-48d5-ba9b-ffaf8c5c6398 service neutron] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:32:16.160403 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6937ce24-1a1a-48d5-ba9b-ffaf8c5c6398 service neutron] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:32:16.166128 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe4595f-1f5a-4713-9f49-56bcd8a70024 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:32:16.166994 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe4595f-1f5a-4713-9f49-56bcd8a70024 admin admin] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:32:16.167459 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe4595f-1f5a-4713-9f49-56bcd8a70024 admin admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:32:16.167876 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe4595f-1f5a-4713-9f49-56bcd8a70024 admin admin] PATH_INFO: `/v3/projects` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:32:16.168900 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe4595f-1f5a-4713-9f49-56bcd8a70024 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:32:16.171008 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe4595f-1f5a-4713-9f49-56bcd8a70024 admin admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:32:16.197369 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1008/2002] 10.222.0.49 () {60 vars in 1152 bytes} [Wed Jun 15 10:32:16 2022] POST /identity/v3/projects => generated 373 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 10:32:16.201469 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd9cc44-0d57-4686-a8a8-83b943b44df4 None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:32:16.213465 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd9cc44-0d57-4686-a8a8-83b943b44df4 None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:32:16.267486 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6869f284-1a3e-41f7-92cc-7ebe2550634c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:32:16.268081 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6869f284-1a3e-41f7-92cc-7ebe2550634c admin admin] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:32:16.268363 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6869f284-1a3e-41f7-92cc-7ebe2550634c admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:32:16.268635 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6869f284-1a3e-41f7-92cc-7ebe2550634c admin admin] PATH_INFO: `/v3/users` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:32:16.269347 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6869f284-1a3e-41f7-92cc-7ebe2550634c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:32:16.271271 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6869f284-1a3e-41f7-92cc-7ebe2550634c admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:32:16.292003 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 995/2003] 10.222.0.49 () {60 vars in 1143 bytes} [Wed Jun 15 10:32:16 2022] POST /identity/v3/users => generated 293 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 10:32:16.295943 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe4595f-1f5a-4713-9f49-56bcd8a70024 admin admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:32:16.306646 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe4595f-1f5a-4713-9f49-56bcd8a70024 admin admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:32:16.311399 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee7d733-b487-4218-8644-c9a7ec9c461f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:32:16.312071 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee7d733-b487-4218-8644-c9a7ec9c461f admin admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:32:16.317987 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee7d733-b487-4218-8644-c9a7ec9c461f admin admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:32:16.318343 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee7d733-b487-4218-8644-c9a7ec9c461f admin admin] PATH_INFO: `/v3/roles` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:32:16.319321 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee7d733-b487-4218-8644-c9a7ec9c461f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:32:16.321139 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee7d733-b487-4218-8644-c9a7ec9c461f admin admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:32:16.333035 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1009/2004] 10.222.0.49 () {58 vars in 1121 bytes} [Wed Jun 15 10:32:16 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 10:32:16.337709 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6869f284-1a3e-41f7-92cc-7ebe2550634c admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:32:16.347756 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6869f284-1a3e-41f7-92cc-7ebe2550634c admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:32:16.353187 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab37626-bc94-4de9-beed-3f69ffc6f823 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:32:16.354375 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab37626-bc94-4de9-beed-3f69ffc6f823 admin admin] REQUEST_METHOD: `PUT` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:32:16.354375 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab37626-bc94-4de9-beed-3f69ffc6f823 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:32:16.354607 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab37626-bc94-4de9-beed-3f69ffc6f823 admin admin] PATH_INFO: `/v3/projects/abb31d897fb8455f905262e723176e4b/users/0fbee98bb3804d79898e0b6995ab0f18/roles/8051174e7a7947d5b991ebd793c2e543` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:32:16.369190 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab37626-bc94-4de9-beed-3f69ffc6f823 admin admin] RBAC: Authorizing `identity:create_grant(project_id=abb31d897fb8455f905262e723176e4b, user_id=0fbee98bb3804d79898e0b6995ab0f18, role_id=8051174e7a7947d5b991ebd793c2e543)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:32:16.371905 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab37626-bc94-4de9-beed-3f69ffc6f823 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:32:16.384359 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 996/2005] 10.222.0.49 () {60 vars in 1482 bytes} [Wed Jun 15 10:32:16 2022] PUT /identity/v3/projects/abb31d897fb8455f905262e723176e4b/users/0fbee98bb3804d79898e0b6995ab0f18/roles/8051174e7a7947d5b991ebd793c2e543 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:32:16.388262 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee7d733-b487-4218-8644-c9a7ec9c461f admin admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:32:16.398687 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee7d733-b487-4218-8644-c9a7ec9c461f admin admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:32:16.454806 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b077864-774a-4c46-96f1-84c305d1c0cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:32:16.455507 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b077864-774a-4c46-96f1-84c305d1c0cf admin admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:32:16.455789 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b077864-774a-4c46-96f1-84c305d1c0cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:32:16.456062 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b077864-774a-4c46-96f1-84c305d1c0cf admin admin] PATH_INFO: `/v3/roles` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:32:16.456778 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b077864-774a-4c46-96f1-84c305d1c0cf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:32:16.458365 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b077864-774a-4c46-96f1-84c305d1c0cf admin admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:32:16.468548 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1010/2006] 10.222.0.49 () {58 vars in 1121 bytes} [Wed Jun 15 10:32:16 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 10:32:16.472659 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab37626-bc94-4de9-beed-3f69ffc6f823 admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:32:16.482703 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab37626-bc94-4de9-beed-3f69ffc6f823 admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:32:16.487308 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f7c120-dd10-4704-a9c1-e139e1efa911 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:32:16.488186 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f7c120-dd10-4704-a9c1-e139e1efa911 admin admin] REQUEST_METHOD: `PUT` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:32:16.488461 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f7c120-dd10-4704-a9c1-e139e1efa911 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:32:16.488729 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f7c120-dd10-4704-a9c1-e139e1efa911 admin admin] PATH_INFO: `/v3/projects/abb31d897fb8455f905262e723176e4b/users/0fbee98bb3804d79898e0b6995ab0f18/roles/8051174e7a7947d5b991ebd793c2e543` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:32:16.490961 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f7c120-dd10-4704-a9c1-e139e1efa911 admin admin] RBAC: Authorizing `identity:create_grant(project_id=abb31d897fb8455f905262e723176e4b, user_id=0fbee98bb3804d79898e0b6995ab0f18, role_id=8051174e7a7947d5b991ebd793c2e543)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:32:16.493610 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f7c120-dd10-4704-a9c1-e139e1efa911 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:32:16.503816 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 997/2007] 10.222.0.49 () {60 vars in 1482 bytes} [Wed Jun 15 10:32:16 2022] PUT /identity/v3/projects/abb31d897fb8455f905262e723176e4b/users/0fbee98bb3804d79898e0b6995ab0f18/roles/8051174e7a7947d5b991ebd793c2e543 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:32:17.156733 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdfb74f-87ee-455b-ab09-d5f774af64a8 None None] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:32:17.157521 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdfb74f-87ee-455b-ab09-d5f774af64a8 None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:32:17.157754 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdfb74f-87ee-455b-ab09-d5f774af64a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:32:17.196429 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.auth.core [None req-2fdfb74f-87ee-455b-ab09-d5f774af64a8 None None] MFA Rules not processed for user `0fbee98bb3804d79898e0b6995ab0f18`. Rule list: `[]` (Enabled: `True`). {{(pid=80119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:32:17.197829 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-2fdfb74f-87ee-455b-ab09-d5f774af64a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:32:17.265113 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:32:17.307431 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1011/2008] 10.222.0.49 () {58 vars in 957 bytes} [Wed Jun 15 10:32:17 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:32:17.319715 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f7c120-dd10-4704-a9c1-e139e1efa911 admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:32:17.325468 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-79f7c120-dd10-4704-a9c1-e139e1efa911 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:32:17.395360 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f7c120-dd10-4704-a9c1-e139e1efa911 admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:32:17.399321 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad198f3a-c115-4fab-9ace-0bb7ed634d11 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20d1327553404b80aecee87c78fcb52b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '559b3e713f5844b6ae37151d56dd419f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:32:17.409038 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad198f3a-c115-4fab-9ace-0bb7ed634d11 service cinder] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:32:17.409038 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad198f3a-c115-4fab-9ace-0bb7ed634d11 service cinder] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:32:17.409038 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad198f3a-c115-4fab-9ace-0bb7ed634d11 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:32:17.432760 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:32:17.477163 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad198f3a-c115-4fab-9ace-0bb7ed634d11 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:32:17.478644 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad198f3a-c115-4fab-9ace-0bb7ed634d11 service cinder] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:32:17.488932 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 998/2009] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Jun 15 10:32:17 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:32:18.859550 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fdfb74f-87ee-455b-ab09-d5f774af64a8 None None] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:32:18.869306 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fdfb74f-87ee-455b-ab09-d5f774af64a8 None None] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:32:18.872902 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ba0f31-fe2a-4ae1-9b21-0580f372e955 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20d1327553404b80aecee87c78fcb52b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '559b3e713f5844b6ae37151d56dd419f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:32:18.873490 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ba0f31-fe2a-4ae1-9b21-0580f372e955 service cinder] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:32:18.873770 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ba0f31-fe2a-4ae1-9b21-0580f372e955 service cinder] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:32:18.874047 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ba0f31-fe2a-4ae1-9b21-0580f372e955 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:32:18.884350 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ba0f31-fe2a-4ae1-9b21-0580f372e955 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:32:18.885813 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ba0f31-fe2a-4ae1-9b21-0580f372e955 service cinder] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:32:18.895996 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1012/2010] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Jun 15 10:32:18 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:32:27.483554 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=18 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:32:28.891394 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=11 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:32:36.004966 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad198f3a-c115-4fab-9ace-0bb7ed634d11 service cinder] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:32:36.006466 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-ad198f3a-c115-4fab-9ace-0bb7ed634d11 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:32:36.129128 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad198f3a-c115-4fab-9ace-0bb7ed634d11 service cinder] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:32:36.136898 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27e8a5e-89a6-4578-b3b3-92a1402d06f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:32:36.138791 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27e8a5e-89a6-4578-b3b3-92a1402d06f9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:32:36.139292 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27e8a5e-89a6-4578-b3b3-92a1402d06f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:32:36.139723 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27e8a5e-89a6-4578-b3b3-92a1402d06f9 admin admin] PATH_INFO: `/v3/users/38e2794bd474425d9d75f21c1929972b` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:32:36.141116 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b27e8a5e-89a6-4578-b3b3-92a1402d06f9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=38e2794bd474425d9d75f21c1929972b)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:32:36.146387 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b27e8a5e-89a6-4578-b3b3-92a1402d06f9 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:32:36.203234 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-b27e8a5e-89a6-4578-b3b3-92a1402d06f9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '38e2794bd474425d9d75f21c1929972b'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:32:36.210519 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-b27e8a5e-89a6-4578-b3b3-92a1402d06f9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '38e2794bd474425d9d75f21c1929972b'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:32:36.211286 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-b27e8a5e-89a6-4578-b3b3-92a1402d06f9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '38e2794bd474425d9d75f21c1929972b'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:32:36.211948 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-b27e8a5e-89a6-4578-b3b3-92a1402d06f9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '38e2794bd474425d9d75f21c1929972b'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:32:36.220395 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-b27e8a5e-89a6-4578-b3b3-92a1402d06f9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '38e2794bd474425d9d75f21c1929972b'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:32:36.239330 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 999/2011] 10.222.0.49 () {58 vars in 1223 bytes} [Wed Jun 15 10:32:36 2022] DELETE /identity/v3/users/38e2794bd474425d9d75f21c1929972b => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:32:36.425446 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ba0f31-fe2a-4ae1-9b21-0580f372e955 service cinder] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:32:36.439700 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ba0f31-fe2a-4ae1-9b21-0580f372e955 service cinder] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:32:36.504850 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95bf89b4-b0dc-4503-b23d-9d41d7c26aeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:32:36.505708 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bf89b4-b0dc-4503-b23d-9d41d7c26aeb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:32:36.505989 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bf89b4-b0dc-4503-b23d-9d41d7c26aeb admin admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:32:36.506263 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bf89b4-b0dc-4503-b23d-9d41d7c26aeb admin admin] PATH_INFO: `/v3/projects/b8df5f1986694e8f9ee6f55ef3995621` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:32:36.508094 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95bf89b4-b0dc-4503-b23d-9d41d7c26aeb admin admin] RBAC: Authorizing `identity:delete_project(project_id=b8df5f1986694e8f9ee6f55ef3995621)` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:32:36.510041 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95bf89b4-b0dc-4503-b23d-9d41d7c26aeb admin admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:32:36.562722 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-95bf89b4-b0dc-4503-b23d-9d41d7c26aeb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b8df5f1986694e8f9ee6f55ef3995621', 'request_id': 'req-95bf89b4-b0dc-4503-b23d-9d41d7c26aeb'} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:32:36.571190 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-95bf89b4-b0dc-4503-b23d-9d41d7c26aeb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b8df5f1986694e8f9ee6f55ef3995621', 'request_id': 'req-95bf89b4-b0dc-4503-b23d-9d41d7c26aeb'} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:32:36.577603 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-95bf89b4-b0dc-4503-b23d-9d41d7c26aeb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b8df5f1986694e8f9ee6f55ef3995621', 'request_id': 'req-95bf89b4-b0dc-4503-b23d-9d41d7c26aeb'} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:32:36.591445 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-95bf89b4-b0dc-4503-b23d-9d41d7c26aeb admin admin] The token cache is being invalidate because project b8df5f1986694e8f9ee6f55ef3995621 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80119) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 10:32:36.591804 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-95bf89b4-b0dc-4503-b23d-9d41d7c26aeb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:32:36.592486 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-95bf89b4-b0dc-4503-b23d-9d41d7c26aeb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:32:36.594125 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1013/2012] 10.222.0.49 () {58 vars in 1232 bytes} [Wed Jun 15 10:32:36 2022] DELETE /identity/v3/projects/b8df5f1986694e8f9ee6f55ef3995621 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:32:36.600212 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ea2ee4-37c4-40ca-8862-b179efee881f None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:32:36.600548 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ea2ee4-37c4-40ca-8862-b179efee881f None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:32:36.600874 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ea2ee4-37c4-40ca-8862-b179efee881f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:32:36.620482 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-b3ea2ee4-37c4-40ca-8862-b179efee881f None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:32:36.621711 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-b3ea2ee4-37c4-40ca-8862-b179efee881f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:32:36.683348 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1000/2013] 10.222.0.49 () {58 vars in 957 bytes} [Wed Jun 15 10:32:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:32:36.691620 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619e5ca5-5d6e-4cb9-a70c-ebd5f567af0e None None] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:32:36.692005 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619e5ca5-5d6e-4cb9-a70c-ebd5f567af0e None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:32:36.692273 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619e5ca5-5d6e-4cb9-a70c-ebd5f567af0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:32:36.711718 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.auth.core [None req-619e5ca5-5d6e-4cb9-a70c-ebd5f567af0e None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:32:36.713473 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-619e5ca5-5d6e-4cb9-a70c-ebd5f567af0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:32:36.721553 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1014/2014] 10.222.0.49 () {58 vars in 957 bytes} [Wed Jun 15 10:32:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:32:36.726026 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ea2ee4-37c4-40ca-8862-b179efee881f None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:32:36.736822 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ea2ee4-37c4-40ca-8862-b179efee881f None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:32:36.741474 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660f785c-cff6-402e-a5db-938e338d0196 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:32:36.742135 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660f785c-cff6-402e-a5db-938e338d0196 admin admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:32:36.742424 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660f785c-cff6-402e-a5db-938e338d0196 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:32:36.742705 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660f785c-cff6-402e-a5db-938e338d0196 admin admin] PATH_INFO: `/v3/domains` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:32:36.743472 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660f785c-cff6-402e-a5db-938e338d0196 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:32:36.743803 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660f785c-cff6-402e-a5db-938e338d0196 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:32:36.745399 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660f785c-cff6-402e-a5db-938e338d0196 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:32:36.758435 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1001/2015] 10.222.0.49 () {58 vars in 1152 bytes} [Wed Jun 15 10:32:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:32:36.763609 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619e5ca5-5d6e-4cb9-a70c-ebd5f567af0e None None] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:32:36.773538 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619e5ca5-5d6e-4cb9-a70c-ebd5f567af0e None None] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:32:36.779505 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91721b8-2c9c-4373-90b7-53b5280e5fb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:32:36.780170 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91721b8-2c9c-4373-90b7-53b5280e5fb4 admin admin] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:32:36.780455 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91721b8-2c9c-4373-90b7-53b5280e5fb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:32:36.780732 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91721b8-2c9c-4373-90b7-53b5280e5fb4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:32:36.781420 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c91721b8-2c9c-4373-90b7-53b5280e5fb4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:32:36.783637 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c91721b8-2c9c-4373-90b7-53b5280e5fb4 admin admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:32:36.808720 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1015/2016] 10.222.0.49 () {60 vars in 1152 bytes} [Wed Jun 15 10:32:36 2022] POST /identity/v3/projects => generated 377 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 10:32:36.812848 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660f785c-cff6-402e-a5db-938e338d0196 admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:32:36.823304 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660f785c-cff6-402e-a5db-938e338d0196 admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:32:36.826862 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:32:36.881140 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c0538b-1da3-40d3-904f-d2564ae00e3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:32:36.881811 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c0538b-1da3-40d3-904f-d2564ae00e3d admin admin] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:32:36.882099 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c0538b-1da3-40d3-904f-d2564ae00e3d admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:32:36.882377 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c0538b-1da3-40d3-904f-d2564ae00e3d admin admin] PATH_INFO: `/v3/users` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:32:36.883119 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c0538b-1da3-40d3-904f-d2564ae00e3d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:32:36.885576 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c0538b-1da3-40d3-904f-d2564ae00e3d admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:32:36.910413 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1002/2017] 10.222.0.49 () {60 vars in 1143 bytes} [Wed Jun 15 10:32:36 2022] POST /identity/v3/users => generated 289 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 10:32:36.911714 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91721b8-2c9c-4373-90b7-53b5280e5fb4 admin admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:32:36.922701 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91721b8-2c9c-4373-90b7-53b5280e5fb4 admin admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:32:36.928021 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12cd9578-90f5-46ca-bf60-6a880077f0c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:32:36.928689 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cd9578-90f5-46ca-bf60-6a880077f0c4 admin admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:32:36.928975 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cd9578-90f5-46ca-bf60-6a880077f0c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:32:36.929256 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cd9578-90f5-46ca-bf60-6a880077f0c4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:32:36.929982 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12cd9578-90f5-46ca-bf60-6a880077f0c4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:32:36.931690 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12cd9578-90f5-46ca-bf60-6a880077f0c4 admin admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:32:36.942975 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1016/2018] 10.222.0.49 () {58 vars in 1121 bytes} [Wed Jun 15 10:32:36 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 10:32:36.947824 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c0538b-1da3-40d3-904f-d2564ae00e3d admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:32:36.957825 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c0538b-1da3-40d3-904f-d2564ae00e3d admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:32:36.963540 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4c2e49-8e43-4ba3-8f4c-f83a8028ecad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:32:36.964281 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4c2e49-8e43-4ba3-8f4c-f83a8028ecad admin admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:32:36.964570 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4c2e49-8e43-4ba3-8f4c-f83a8028ecad admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:32:36.964847 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4c2e49-8e43-4ba3-8f4c-f83a8028ecad admin admin] PATH_INFO: `/v3/roles` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:32:36.965581 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f4c2e49-8e43-4ba3-8f4c-f83a8028ecad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:32:36.967281 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f4c2e49-8e43-4ba3-8f4c-f83a8028ecad admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:32:36.981475 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1003/2019] 10.222.0.49 () {58 vars in 1121 bytes} [Wed Jun 15 10:32:36 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 10:32:36.982717 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12cd9578-90f5-46ca-bf60-6a880077f0c4 admin admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:32:36.992788 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12cd9578-90f5-46ca-bf60-6a880077f0c4 admin admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:32:36.997898 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479df87c-1d26-4557-b916-7248d3378cc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:32:36.998776 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479df87c-1d26-4557-b916-7248d3378cc5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:32:36.999063 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479df87c-1d26-4557-b916-7248d3378cc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:32:36.999366 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479df87c-1d26-4557-b916-7248d3378cc5 admin admin] PATH_INFO: `/v3/projects/d5320ba7f73d40cb9a6d16225d6660c7/users/9ca22364db8147f2a14f3a0c77f545de/roles/6876e57cc51c448097ab109d35b2a3da` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:32:37.015221 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-479df87c-1d26-4557-b916-7248d3378cc5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d5320ba7f73d40cb9a6d16225d6660c7, user_id=9ca22364db8147f2a14f3a0c77f545de, role_id=6876e57cc51c448097ab109d35b2a3da)` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:32:37.017789 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-479df87c-1d26-4557-b916-7248d3378cc5 admin admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:32:37.031266 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1017/2020] 10.222.0.49 () {60 vars in 1482 bytes} [Wed Jun 15 10:32:36 2022] PUT /identity/v3/projects/d5320ba7f73d40cb9a6d16225d6660c7/users/9ca22364db8147f2a14f3a0c77f545de/roles/6876e57cc51c448097ab109d35b2a3da => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:32:37.036762 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add6b034-fc6f-4ccf-bc53-ce6429301e26 None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:32:37.037242 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add6b034-fc6f-4ccf-bc53-ce6429301e26 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:32:37.037349 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add6b034-fc6f-4ccf-bc53-ce6429301e26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:32:37.056394 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-add6b034-fc6f-4ccf-bc53-ce6429301e26 None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:32:37.057593 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-add6b034-fc6f-4ccf-bc53-ce6429301e26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:32:37.116484 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1004/2021] 10.222.0.49 () {58 vars in 957 bytes} [Wed Jun 15 10:32:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 10:32:37.126539 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479df87c-1d26-4557-b916-7248d3378cc5 admin admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:32:37.128545 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-479df87c-1d26-4557-b916-7248d3378cc5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:32:37.159521 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:32:37.189015 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479df87c-1d26-4557-b916-7248d3378cc5 admin admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:32:37.194344 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31dde60d-654b-424c-ab24-d8049ea31b9f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0cb44a17d205431793e9a8791e477a32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '559b3e713f5844b6ae37151d56dd419f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:32:37.195006 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31dde60d-654b-424c-ab24-d8049ea31b9f service neutron] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:32:37.195322 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31dde60d-654b-424c-ab24-d8049ea31b9f service neutron] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:32:37.196028 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31dde60d-654b-424c-ab24-d8049ea31b9f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:32:37.216732 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31dde60d-654b-424c-ab24-d8049ea31b9f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:32:37.218503 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31dde60d-654b-424c-ab24-d8049ea31b9f service neutron] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:32:37.229050 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1018/2022] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Jun 15 10:32:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:32:40.674816 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add6b034-fc6f-4ccf-bc53-ce6429301e26 None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:32:40.676547 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-add6b034-fc6f-4ccf-bc53-ce6429301e26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:32:40.689205 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add6b034-fc6f-4ccf-bc53-ce6429301e26 None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:32:40.697242 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d7fd48-9531-40f5-bba1-39ec968d8344 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:32:40.698372 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d7fd48-9531-40f5-bba1-39ec968d8344 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:32:40.698830 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d7fd48-9531-40f5-bba1-39ec968d8344 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:32:40.699309 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d7fd48-9531-40f5-bba1-39ec968d8344 admin admin] PATH_INFO: `/v3/users/de3df1675add45558d3fd4183dd1785e` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:32:40.700731 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d7fd48-9531-40f5-bba1-39ec968d8344 admin admin] RBAC: Authorizing `identity:delete_user(user_id=de3df1675add45558d3fd4183dd1785e)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:32:40.702982 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d7fd48-9531-40f5-bba1-39ec968d8344 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:32:40.759233 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-f4d7fd48-9531-40f5-bba1-39ec968d8344 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'de3df1675add45558d3fd4183dd1785e'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:32:40.773309 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-f4d7fd48-9531-40f5-bba1-39ec968d8344 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'de3df1675add45558d3fd4183dd1785e'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:32:40.774380 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-f4d7fd48-9531-40f5-bba1-39ec968d8344 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'de3df1675add45558d3fd4183dd1785e'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:32:40.775220 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-f4d7fd48-9531-40f5-bba1-39ec968d8344 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'de3df1675add45558d3fd4183dd1785e'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:32:40.785160 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-f4d7fd48-9531-40f5-bba1-39ec968d8344 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'de3df1675add45558d3fd4183dd1785e'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:32:40.811385 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:32:40.820660 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1005/2023] 10.222.0.49 () {58 vars in 1223 bytes} [Wed Jun 15 10:32:40 2022] DELETE /identity/v3/users/de3df1675add45558d3fd4183dd1785e => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:32:40.823042 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9ad04a-e587-448d-a4fd-5dd1076caee7 None None] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:32:40.823557 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9ad04a-e587-448d-a4fd-5dd1076caee7 None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:32:40.823659 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9ad04a-e587-448d-a4fd-5dd1076caee7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:32:40.845966 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.auth.core [None req-ea9ad04a-e587-448d-a4fd-5dd1076caee7 None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:32:40.847879 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-ea9ad04a-e587-448d-a4fd-5dd1076caee7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:32:40.920896 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1019/2024] 10.222.0.49 () {58 vars in 957 bytes} [Wed Jun 15 10:32:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:32:40.932902 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d7fd48-9531-40f5-bba1-39ec968d8344 admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:32:40.934595 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-f4d7fd48-9531-40f5-bba1-39ec968d8344 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:32:41.062472 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d7fd48-9531-40f5-bba1-39ec968d8344 admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:32:41.067382 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b7c61c-c398-419f-8bc8-05e14e558b5e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0cb44a17d205431793e9a8791e477a32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '559b3e713f5844b6ae37151d56dd419f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:32:41.068166 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b7c61c-c398-419f-8bc8-05e14e558b5e service neutron] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:32:41.068459 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b7c61c-c398-419f-8bc8-05e14e558b5e service neutron] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:32:41.068795 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b7c61c-c398-419f-8bc8-05e14e558b5e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:32:41.081732 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b7c61c-c398-419f-8bc8-05e14e558b5e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:32:41.083418 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b7c61c-c398-419f-8bc8-05e14e558b5e service neutron] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:32:41.097008 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1006/2025] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Jun 15 10:32:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:32:41.407994 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9ad04a-e587-448d-a4fd-5dd1076caee7 None None] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:32:41.409465 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-ea9ad04a-e587-448d-a4fd-5dd1076caee7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:32:41.428525 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9ad04a-e587-448d-a4fd-5dd1076caee7 None None] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:32:41.438202 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2704c0f1-90b0-4be6-8bae-f8f4f35bd33d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:32:41.440166 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2704c0f1-90b0-4be6-8bae-f8f4f35bd33d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:32:41.440464 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2704c0f1-90b0-4be6-8bae-f8f4f35bd33d admin admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:32:41.440737 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2704c0f1-90b0-4be6-8bae-f8f4f35bd33d admin admin] PATH_INFO: `/v3/projects/3f66feeddeba414296d6f0a04e97457a` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:32:41.442876 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2704c0f1-90b0-4be6-8bae-f8f4f35bd33d admin admin] RBAC: Authorizing `identity:delete_project(project_id=3f66feeddeba414296d6f0a04e97457a)` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:32:41.446766 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2704c0f1-90b0-4be6-8bae-f8f4f35bd33d admin admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:32:41.558919 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-2704c0f1-90b0-4be6-8bae-f8f4f35bd33d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3f66feeddeba414296d6f0a04e97457a', 'request_id': 'req-2704c0f1-90b0-4be6-8bae-f8f4f35bd33d'} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:32:41.587267 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-2704c0f1-90b0-4be6-8bae-f8f4f35bd33d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3f66feeddeba414296d6f0a04e97457a', 'request_id': 'req-2704c0f1-90b0-4be6-8bae-f8f4f35bd33d'} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:32:41.598055 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-2704c0f1-90b0-4be6-8bae-f8f4f35bd33d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3f66feeddeba414296d6f0a04e97457a', 'request_id': 'req-2704c0f1-90b0-4be6-8bae-f8f4f35bd33d'} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:32:41.620287 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-2704c0f1-90b0-4be6-8bae-f8f4f35bd33d admin admin] The token cache is being invalidate because project 3f66feeddeba414296d6f0a04e97457a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80119) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 10:32:41.621120 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-2704c0f1-90b0-4be6-8bae-f8f4f35bd33d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:32:41.622902 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-2704c0f1-90b0-4be6-8bae-f8f4f35bd33d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:32:41.626769 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1020/2026] 10.222.0.49 () {58 vars in 1232 bytes} [Wed Jun 15 10:32:41 2022] DELETE /identity/v3/projects/3f66feeddeba414296d6f0a04e97457a => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:32:41.631568 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c577b0-2223-4582-b869-e1c5c225bae1 None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:32:41.632045 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c577b0-2223-4582-b869-e1c5c225bae1 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:32:41.632476 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c577b0-2223-4582-b869-e1c5c225bae1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:32:41.670959 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-d1c577b0-2223-4582-b869-e1c5c225bae1 None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:32:41.672427 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-d1c577b0-2223-4582-b869-e1c5c225bae1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:32:41.835348 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14083b20-09af-4aa6-8c13-4310276102d5 None None] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:32:41.835840 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14083b20-09af-4aa6-8c13-4310276102d5 None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:32:41.835916 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14083b20-09af-4aa6-8c13-4310276102d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:32:41.853312 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1007/2027] 10.222.0.49 () {58 vars in 957 bytes} [Wed Jun 15 10:32:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:32:41.932036 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.auth.core [None req-14083b20-09af-4aa6-8c13-4310276102d5 None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:32:41.933963 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-14083b20-09af-4aa6-8c13-4310276102d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:32:41.960567 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1021/2028] 10.222.0.49 () {58 vars in 957 bytes} [Wed Jun 15 10:32:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:32:41.965308 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c577b0-2223-4582-b869-e1c5c225bae1 None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:32:41.984845 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c577b0-2223-4582-b869-e1c5c225bae1 None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:32:41.999770 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12608ed-9a24-4309-b3fe-f2470cc195d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:32:42.000784 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12608ed-9a24-4309-b3fe-f2470cc195d4 admin admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:32:42.001236 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12608ed-9a24-4309-b3fe-f2470cc195d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:32:42.001703 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12608ed-9a24-4309-b3fe-f2470cc195d4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:32:42.002670 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12608ed-9a24-4309-b3fe-f2470cc195d4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:32:42.003240 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12608ed-9a24-4309-b3fe-f2470cc195d4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:32:42.007851 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12608ed-9a24-4309-b3fe-f2470cc195d4 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:32:42.034954 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1008/2029] 10.222.0.49 () {58 vars in 1152 bytes} [Wed Jun 15 10:32:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:32:42.039275 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14083b20-09af-4aa6-8c13-4310276102d5 None None] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:32:42.055105 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14083b20-09af-4aa6-8c13-4310276102d5 None None] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:32:42.067813 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7efaaecd-b10e-4831-bf96-f57ed610b1e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:32:42.068671 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efaaecd-b10e-4831-bf96-f57ed610b1e4 admin admin] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:32:42.069193 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efaaecd-b10e-4831-bf96-f57ed610b1e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:32:42.069533 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efaaecd-b10e-4831-bf96-f57ed610b1e4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:32:42.070309 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7efaaecd-b10e-4831-bf96-f57ed610b1e4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:32:42.072409 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7efaaecd-b10e-4831-bf96-f57ed610b1e4 admin admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:32:42.120637 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1022/2030] 10.222.0.49 () {60 vars in 1152 bytes} [Wed Jun 15 10:32:42 2022] POST /identity/v3/projects => generated 367 bytes in 82 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 10:32:42.124859 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12608ed-9a24-4309-b3fe-f2470cc195d4 admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:32:42.145026 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12608ed-9a24-4309-b3fe-f2470cc195d4 admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:32:42.286845 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5e441b-1a53-494e-8110-4901e9ec4517 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:32:42.287589 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5e441b-1a53-494e-8110-4901e9ec4517 admin admin] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:32:42.287871 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5e441b-1a53-494e-8110-4901e9ec4517 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:32:42.288148 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5e441b-1a53-494e-8110-4901e9ec4517 admin admin] PATH_INFO: `/v3/users` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:32:42.288852 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a5e441b-1a53-494e-8110-4901e9ec4517 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:32:42.291539 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a5e441b-1a53-494e-8110-4901e9ec4517 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:32:42.332628 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9b4944-8502-45de-88d8-ca1ceb7682b4 None None] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:32:42.333073 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9b4944-8502-45de-88d8-ca1ceb7682b4 None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:32:42.333455 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9b4944-8502-45de-88d8-ca1ceb7682b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:32:42.341356 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1009/2031] 10.222.0.49 () {60 vars in 1143 bytes} [Wed Jun 15 10:32:42 2022] POST /identity/v3/users => generated 284 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 10:32:42.342670 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5e441b-1a53-494e-8110-4901e9ec4517 admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:32:42.358111 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5e441b-1a53-494e-8110-4901e9ec4517 admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:32:42.363223 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc482efa-3f45-421c-99d0-a7f14803d9be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:32:42.363911 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc482efa-3f45-421c-99d0-a7f14803d9be admin admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:32:42.364196 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc482efa-3f45-421c-99d0-a7f14803d9be admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:32:42.365140 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc482efa-3f45-421c-99d0-a7f14803d9be admin admin] PATH_INFO: `/v3/roles` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:32:42.365893 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc482efa-3f45-421c-99d0-a7f14803d9be admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:32:42.367549 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc482efa-3f45-421c-99d0-a7f14803d9be admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:32:42.382751 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.auth.core [None req-ee9b4944-8502-45de-88d8-ca1ceb7682b4 None None] MFA Rules not processed for user `9ca22364db8147f2a14f3a0c77f545de`. Rule list: `[]` (Enabled: `True`). {{(pid=80119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:32:42.384262 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1010/2032] 10.222.0.49 () {58 vars in 1121 bytes} [Wed Jun 15 10:32:42 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 10:32:42.385483 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc482efa-3f45-421c-99d0-a7f14803d9be admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:32:42.387553 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-ee9b4944-8502-45de-88d8-ca1ceb7682b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:32:42.413213 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:32:42.424161 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc482efa-3f45-421c-99d0-a7f14803d9be admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:32:42.443115 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f13e8b7-6bc1-459a-8bb5-2585fd72363b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:32:42.444066 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f13e8b7-6bc1-459a-8bb5-2585fd72363b admin admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:32:42.444423 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f13e8b7-6bc1-459a-8bb5-2585fd72363b admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:32:42.445461 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f13e8b7-6bc1-459a-8bb5-2585fd72363b admin admin] PATH_INFO: `/v3/roles` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:32:42.446423 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f13e8b7-6bc1-459a-8bb5-2585fd72363b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:32:42.448518 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f13e8b7-6bc1-459a-8bb5-2585fd72363b admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:32:42.450046 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:32:42.485078 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1011/2033] 10.222.0.49 () {58 vars in 1121 bytes} [Wed Jun 15 10:32:42 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 10:32:42.490519 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f13e8b7-6bc1-459a-8bb5-2585fd72363b admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:32:42.510982 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f13e8b7-6bc1-459a-8bb5-2585fd72363b admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:32:42.512939 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1023/2034] 10.222.0.49 () {58 vars in 957 bytes} [Wed Jun 15 10:32:42 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 10:32:42.516670 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6643c3-5a9c-426d-a1b4-2c2855be88e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:32:42.517604 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6643c3-5a9c-426d-a1b4-2c2855be88e6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:32:42.517889 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6643c3-5a9c-426d-a1b4-2c2855be88e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:32:42.518172 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6643c3-5a9c-426d-a1b4-2c2855be88e6 admin admin] PATH_INFO: `/v3/projects/f655ca9cf258481d8fe8b85af5c80392/users/5c39c8958eb64553bfad994b4eb21db8/roles/6876e57cc51c448097ab109d35b2a3da` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:32:42.529707 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9b4944-8502-45de-88d8-ca1ceb7682b4 None None] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:32:42.531299 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-ee9b4944-8502-45de-88d8-ca1ceb7682b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:32:42.535840 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6643c3-5a9c-426d-a1b4-2c2855be88e6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f655ca9cf258481d8fe8b85af5c80392, user_id=5c39c8958eb64553bfad994b4eb21db8, role_id=6876e57cc51c448097ab109d35b2a3da)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:32:42.539599 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6643c3-5a9c-426d-a1b4-2c2855be88e6 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:32:42.558206 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1012/2035] 10.222.0.49 () {60 vars in 1482 bytes} [Wed Jun 15 10:32:42 2022] PUT /identity/v3/projects/f655ca9cf258481d8fe8b85af5c80392/users/5c39c8958eb64553bfad994b4eb21db8/roles/6876e57cc51c448097ab109d35b2a3da => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:32:42.570314 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59c6912-433e-4c24-bb6a-961de50fc073 None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:32:42.570606 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59c6912-433e-4c24-bb6a-961de50fc073 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:32:42.571072 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59c6912-433e-4c24-bb6a-961de50fc073 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:32:42.604825 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-b59c6912-433e-4c24-bb6a-961de50fc073 None None] MFA Rules not processed for user `5c39c8958eb64553bfad994b4eb21db8`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:32:42.606379 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-b59c6912-433e-4c24-bb6a-961de50fc073 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:32:42.621999 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9b4944-8502-45de-88d8-ca1ceb7682b4 None None] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:32:42.675757 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1013/2036] 10.222.0.49 () {58 vars in 957 bytes} [Wed Jun 15 10:32:42 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:32:42.687369 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59c6912-433e-4c24-bb6a-961de50fc073 None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:32:42.688789 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-b59c6912-433e-4c24-bb6a-961de50fc073 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:32:42.700125 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfeb013f-fad1-4fdb-8f92-6a2933eef59f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ac54528370d4747b6e8777fba9d0c82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '559b3e713f5844b6ae37151d56dd419f', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:32:42.700804 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfeb013f-fad1-4fdb-8f92-6a2933eef59f service nova] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:32:42.701090 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfeb013f-fad1-4fdb-8f92-6a2933eef59f service nova] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:32:42.701368 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfeb013f-fad1-4fdb-8f92-6a2933eef59f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:32:42.748958 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59c6912-433e-4c24-bb6a-961de50fc073 None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:32:42.752761 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3698b86-3f54-4ad8-9c77-0124956f502c service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91f2fe3be62e49548eae208e57bf2a93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '559b3e713f5844b6ae37151d56dd419f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:32:42.753418 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3698b86-3f54-4ad8-9c77-0124956f502c service glance] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:32:42.753701 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3698b86-3f54-4ad8-9c77-0124956f502c service glance] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:32:42.753959 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=100 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:32:42.754088 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3698b86-3f54-4ad8-9c77-0124956f502c service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:32:42.763018 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3698b86-3f54-4ad8-9c77-0124956f502c service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:32:42.764099 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfeb013f-fad1-4fdb-8f92-6a2933eef59f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:32:42.764504 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3698b86-3f54-4ad8-9c77-0124956f502c service glance] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:32:42.766015 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfeb013f-fad1-4fdb-8f92-6a2933eef59f service nova] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:32:42.779495 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1014/2037] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Jun 15 10:32:42 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:32:42.780112 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1024/2038] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Jun 15 10:32:42 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 251 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:32:44.216816 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d61bf48-bd4b-4b17-96a4-f5d8911a9be7 None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:32:44.217306 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d61bf48-bd4b-4b17-96a4-f5d8911a9be7 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:32:44.217386 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d61bf48-bd4b-4b17-96a4-f5d8911a9be7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:32:44.236957 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-2d61bf48-bd4b-4b17-96a4-f5d8911a9be7 None None] MFA Rules not processed for user `9ca22364db8147f2a14f3a0c77f545de`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:32:44.238267 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-2d61bf48-bd4b-4b17-96a4-f5d8911a9be7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:32:44.248085 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1015/2039] 10.222.0.49 () {58 vars in 957 bytes} [Wed Jun 15 10:32:44 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:32:48.334625 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfeb013f-fad1-4fdb-8f92-6a2933eef59f service nova] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:32:48.337604 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-bfeb013f-fad1-4fdb-8f92-6a2933eef59f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:32:48.440849 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfeb013f-fad1-4fdb-8f92-6a2933eef59f service nova] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:32:48.447323 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831d3021-7893-416a-8c94-5d8a827de92c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:32:48.448761 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831d3021-7893-416a-8c94-5d8a827de92c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:32:48.449243 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831d3021-7893-416a-8c94-5d8a827de92c admin admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:32:48.449493 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831d3021-7893-416a-8c94-5d8a827de92c admin admin] PATH_INFO: `/v3/users/40fd04a3958041d398b14cf5e515214c` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:32:48.452297 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-831d3021-7893-416a-8c94-5d8a827de92c admin admin] RBAC: Authorizing `identity:delete_user(user_id=40fd04a3958041d398b14cf5e515214c)` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:32:48.453986 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-831d3021-7893-416a-8c94-5d8a827de92c admin admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:32:48.595928 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-831d3021-7893-416a-8c94-5d8a827de92c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '40fd04a3958041d398b14cf5e515214c'} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:32:48.607122 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-831d3021-7893-416a-8c94-5d8a827de92c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '40fd04a3958041d398b14cf5e515214c'} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:32:48.615534 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-831d3021-7893-416a-8c94-5d8a827de92c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '40fd04a3958041d398b14cf5e515214c'} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:32:48.616853 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-831d3021-7893-416a-8c94-5d8a827de92c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '40fd04a3958041d398b14cf5e515214c'} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:32:48.638628 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-831d3021-7893-416a-8c94-5d8a827de92c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '40fd04a3958041d398b14cf5e515214c'} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:32:48.676650 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1025/2040] 10.222.0.49 () {58 vars in 1223 bytes} [Wed Jun 15 10:32:48 2022] DELETE /identity/v3/users/40fd04a3958041d398b14cf5e515214c => generated 0 bytes in 343 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:32:48.926275 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d61bf48-bd4b-4b17-96a4-f5d8911a9be7 None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:32:48.939261 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d61bf48-bd4b-4b17-96a4-f5d8911a9be7 None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:32:49.010971 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:32:49.017574 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d376dfc3-f0b4-4f54-8f06-321851bf47ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:32:49.018735 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d376dfc3-f0b4-4f54-8f06-321851bf47ed admin admin] REQUEST_METHOD: `DELETE` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:32:49.019170 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d376dfc3-f0b4-4f54-8f06-321851bf47ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:32:49.019563 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d376dfc3-f0b4-4f54-8f06-321851bf47ed admin admin] PATH_INFO: `/v3/projects/75f96c492c964ea197288dc60d5d60a7` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:32:49.021083 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d376dfc3-f0b4-4f54-8f06-321851bf47ed admin admin] RBAC: Authorizing `identity:delete_project(project_id=75f96c492c964ea197288dc60d5d60a7)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:32:49.023502 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d376dfc3-f0b4-4f54-8f06-321851bf47ed admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:32:49.134939 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-d376dfc3-f0b4-4f54-8f06-321851bf47ed admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '75f96c492c964ea197288dc60d5d60a7', 'request_id': 'req-d376dfc3-f0b4-4f54-8f06-321851bf47ed'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:32:49.147586 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-d376dfc3-f0b4-4f54-8f06-321851bf47ed admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '75f96c492c964ea197288dc60d5d60a7', 'request_id': 'req-d376dfc3-f0b4-4f54-8f06-321851bf47ed'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:32:49.156457 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-d376dfc3-f0b4-4f54-8f06-321851bf47ed admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '75f96c492c964ea197288dc60d5d60a7', 'request_id': 'req-d376dfc3-f0b4-4f54-8f06-321851bf47ed'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:32:49.178765 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-d376dfc3-f0b4-4f54-8f06-321851bf47ed admin admin] The token cache is being invalidate because project 75f96c492c964ea197288dc60d5d60a7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80120) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 10:32:49.179559 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-d376dfc3-f0b4-4f54-8f06-321851bf47ed admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:32:49.180727 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-d376dfc3-f0b4-4f54-8f06-321851bf47ed admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:32:49.182856 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1016/2041] 10.222.0.49 () {58 vars in 1232 bytes} [Wed Jun 15 10:32:48 2022] DELETE /identity/v3/projects/75f96c492c964ea197288dc60d5d60a7 => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:32:49.193118 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831d3021-7893-416a-8c94-5d8a827de92c admin admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:32:49.194966 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-831d3021-7893-416a-8c94-5d8a827de92c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:32:49.209750 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831d3021-7893-416a-8c94-5d8a827de92c admin admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:32:49.273580 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852aa1e6-c4b1-4bb7-8156-f0958318f052 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:32:49.274496 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852aa1e6-c4b1-4bb7-8156-f0958318f052 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:32:49.274805 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852aa1e6-c4b1-4bb7-8156-f0958318f052 admin admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:32:49.275075 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852aa1e6-c4b1-4bb7-8156-f0958318f052 admin admin] PATH_INFO: `/v3/users/0fbee98bb3804d79898e0b6995ab0f18` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:32:49.276300 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-852aa1e6-c4b1-4bb7-8156-f0958318f052 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0fbee98bb3804d79898e0b6995ab0f18)` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:32:49.279146 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-852aa1e6-c4b1-4bb7-8156-f0958318f052 admin admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:32:49.292966 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:32:49.337346 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-852aa1e6-c4b1-4bb7-8156-f0958318f052 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0fbee98bb3804d79898e0b6995ab0f18'} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:32:49.344631 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-852aa1e6-c4b1-4bb7-8156-f0958318f052 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0fbee98bb3804d79898e0b6995ab0f18'} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:32:49.345379 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-852aa1e6-c4b1-4bb7-8156-f0958318f052 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0fbee98bb3804d79898e0b6995ab0f18'} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:32:49.346103 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-852aa1e6-c4b1-4bb7-8156-f0958318f052 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0fbee98bb3804d79898e0b6995ab0f18'} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:32:49.353811 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-852aa1e6-c4b1-4bb7-8156-f0958318f052 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0fbee98bb3804d79898e0b6995ab0f18'} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:32:49.374338 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1026/2042] 10.222.0.49 () {58 vars in 1223 bytes} [Wed Jun 15 10:32:49 2022] DELETE /identity/v3/users/0fbee98bb3804d79898e0b6995ab0f18 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:32:49.628993 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d376dfc3-f0b4-4f54-8f06-321851bf47ed admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:32:49.630791 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-d376dfc3-f0b4-4f54-8f06-321851bf47ed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:32:49.653644 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d376dfc3-f0b4-4f54-8f06-321851bf47ed admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:32:49.763239 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db0f509-f5f9-4a5e-bd99-742181db0567 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:32:49.771852 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db0f509-f5f9-4a5e-bd99-742181db0567 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:32:49.772365 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db0f509-f5f9-4a5e-bd99-742181db0567 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:32:49.772890 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db0f509-f5f9-4a5e-bd99-742181db0567 admin admin] PATH_INFO: `/v3/projects/abb31d897fb8455f905262e723176e4b` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:32:49.774833 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db0f509-f5f9-4a5e-bd99-742181db0567 admin admin] RBAC: Authorizing `identity:delete_project(project_id=abb31d897fb8455f905262e723176e4b)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:32:49.776993 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db0f509-f5f9-4a5e-bd99-742181db0567 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:32:49.861683 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-0db0f509-f5f9-4a5e-bd99-742181db0567 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'abb31d897fb8455f905262e723176e4b', 'request_id': 'req-0db0f509-f5f9-4a5e-bd99-742181db0567'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:32:49.871343 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-0db0f509-f5f9-4a5e-bd99-742181db0567 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'abb31d897fb8455f905262e723176e4b', 'request_id': 'req-0db0f509-f5f9-4a5e-bd99-742181db0567'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:32:49.878790 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-0db0f509-f5f9-4a5e-bd99-742181db0567 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'abb31d897fb8455f905262e723176e4b', 'request_id': 'req-0db0f509-f5f9-4a5e-bd99-742181db0567'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:32:49.891666 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-0db0f509-f5f9-4a5e-bd99-742181db0567 admin admin] The token cache is being invalidate because project abb31d897fb8455f905262e723176e4b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80120) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 10:32:49.892005 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-0db0f509-f5f9-4a5e-bd99-742181db0567 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:32:49.895487 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-0db0f509-f5f9-4a5e-bd99-742181db0567 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:32:49.897745 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1017/2043] 10.222.0.49 () {58 vars in 1232 bytes} [Wed Jun 15 10:32:49 2022] DELETE /identity/v3/projects/abb31d897fb8455f905262e723176e4b => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:32:49.903928 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf44839-09dd-4cc5-8585-f32cf46bd058 None None] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:32:49.904452 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf44839-09dd-4cc5-8585-f32cf46bd058 None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:32:49.904877 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf44839-09dd-4cc5-8585-f32cf46bd058 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:32:49.929438 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.auth.core [None req-5bf44839-09dd-4cc5-8585-f32cf46bd058 None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:32:49.930919 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-5bf44839-09dd-4cc5-8585-f32cf46bd058 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:32:50.002163 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1027/2044] 10.222.0.49 () {58 vars in 957 bytes} [Wed Jun 15 10:32:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:32:50.011065 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c272e6-5bd0-47f6-9217-ee5234942c53 None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:32:50.011622 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c272e6-5bd0-47f6-9217-ee5234942c53 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:32:50.011889 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c272e6-5bd0-47f6-9217-ee5234942c53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:32:50.031111 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-b0c272e6-5bd0-47f6-9217-ee5234942c53 None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:32:50.032591 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-b0c272e6-5bd0-47f6-9217-ee5234942c53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:32:50.041928 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1018/2045] 10.222.0.49 () {58 vars in 957 bytes} [Wed Jun 15 10:32:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:32:50.046253 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf44839-09dd-4cc5-8585-f32cf46bd058 None None] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:32:50.056435 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf44839-09dd-4cc5-8585-f32cf46bd058 None None] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:32:50.061029 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0fa0b9-ac8d-4c8c-b70d-319f965a3cd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:32:50.061680 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0fa0b9-ac8d-4c8c-b70d-319f965a3cd5 admin admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:32:50.061977 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0fa0b9-ac8d-4c8c-b70d-319f965a3cd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:32:50.062252 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0fa0b9-ac8d-4c8c-b70d-319f965a3cd5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:32:50.063015 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0fa0b9-ac8d-4c8c-b70d-319f965a3cd5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:32:50.063393 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0fa0b9-ac8d-4c8c-b70d-319f965a3cd5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:32:50.065129 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0fa0b9-ac8d-4c8c-b70d-319f965a3cd5 admin admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:32:50.076686 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1028/2046] 10.222.0.49 () {58 vars in 1152 bytes} [Wed Jun 15 10:32:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 10:32:50.081153 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c272e6-5bd0-47f6-9217-ee5234942c53 None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:32:50.090942 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c272e6-5bd0-47f6-9217-ee5234942c53 None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:32:50.095710 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a83f92-6c8c-47c8-9df9-2ce82ebc1414 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:32:50.096363 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a83f92-6c8c-47c8-9df9-2ce82ebc1414 admin admin] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:32:50.096649 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a83f92-6c8c-47c8-9df9-2ce82ebc1414 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:32:50.096927 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a83f92-6c8c-47c8-9df9-2ce82ebc1414 admin admin] PATH_INFO: `/v3/projects` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:32:50.097625 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a83f92-6c8c-47c8-9df9-2ce82ebc1414 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:32:50.099560 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a83f92-6c8c-47c8-9df9-2ce82ebc1414 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:32:50.108203 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:32:50.127393 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1019/2047] 10.222.0.49 () {60 vars in 1152 bytes} [Wed Jun 15 10:32:50 2022] POST /identity/v3/projects => generated 371 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 10:32:50.131485 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0fa0b9-ac8d-4c8c-b70d-319f965a3cd5 admin admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:32:50.142033 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0fa0b9-ac8d-4c8c-b70d-319f965a3cd5 admin admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:32:50.198202 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b415d77-c14a-433d-87b5-3cbeea94efb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:32:50.198877 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b415d77-c14a-433d-87b5-3cbeea94efb6 admin admin] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:32:50.199193 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b415d77-c14a-433d-87b5-3cbeea94efb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:32:50.199472 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b415d77-c14a-433d-87b5-3cbeea94efb6 admin admin] PATH_INFO: `/v3/users` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:32:50.200183 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b415d77-c14a-433d-87b5-3cbeea94efb6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:32:50.202099 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b415d77-c14a-433d-87b5-3cbeea94efb6 admin admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:32:50.228232 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1029/2048] 10.222.0.49 () {60 vars in 1143 bytes} [Wed Jun 15 10:32:50 2022] POST /identity/v3/users => generated 286 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 10:32:50.229961 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a83f92-6c8c-47c8-9df9-2ce82ebc1414 admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:32:50.240777 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a83f92-6c8c-47c8-9df9-2ce82ebc1414 admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:32:50.247241 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d11703-6b04-4103-8001-cbdf8f383538 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:32:50.248246 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d11703-6b04-4103-8001-cbdf8f383538 admin admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:32:50.248737 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d11703-6b04-4103-8001-cbdf8f383538 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:32:50.249176 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d11703-6b04-4103-8001-cbdf8f383538 admin admin] PATH_INFO: `/v3/roles` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:32:50.250094 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d11703-6b04-4103-8001-cbdf8f383538 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:32:50.251760 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d11703-6b04-4103-8001-cbdf8f383538 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:32:50.267270 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1020/2049] 10.222.0.49 () {58 vars in 1121 bytes} [Wed Jun 15 10:32:50 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 10:32:50.269575 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b415d77-c14a-433d-87b5-3cbeea94efb6 admin admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:32:50.276753 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:DELETE=10,keystone:SELECT=86,keystone:INSERT=4 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:32:50.281770 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b415d77-c14a-433d-87b5-3cbeea94efb6 admin admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:32:50.289057 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715484d5-cd90-49c9-9043-7369c58153bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:32:50.290292 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715484d5-cd90-49c9-9043-7369c58153bb admin admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:32:50.290641 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715484d5-cd90-49c9-9043-7369c58153bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:32:50.290943 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715484d5-cd90-49c9-9043-7369c58153bb admin admin] PATH_INFO: `/v3/roles` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:32:50.291713 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-715484d5-cd90-49c9-9043-7369c58153bb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:32:50.293377 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-715484d5-cd90-49c9-9043-7369c58153bb admin admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:32:50.304540 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1030/2050] 10.222.0.49 () {58 vars in 1121 bytes} [Wed Jun 15 10:32:50 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 15 10:32:50.309236 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d11703-6b04-4103-8001-cbdf8f383538 admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:32:50.319602 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d11703-6b04-4103-8001-cbdf8f383538 admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:32:50.324537 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb569a8-491c-46f0-a02b-ac44d54f2c02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:32:50.325440 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb569a8-491c-46f0-a02b-ac44d54f2c02 admin admin] REQUEST_METHOD: `PUT` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:32:50.325728 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb569a8-491c-46f0-a02b-ac44d54f2c02 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:32:50.326006 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb569a8-491c-46f0-a02b-ac44d54f2c02 admin admin] PATH_INFO: `/v3/projects/02cd089ef0c44ebda9956d2be371fa32/users/184e49aec8cb4c6493c14711a7cdc078/roles/6876e57cc51c448097ab109d35b2a3da` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:32:50.342243 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb569a8-491c-46f0-a02b-ac44d54f2c02 admin admin] RBAC: Authorizing `identity:create_grant(project_id=02cd089ef0c44ebda9956d2be371fa32, user_id=184e49aec8cb4c6493c14711a7cdc078, role_id=6876e57cc51c448097ab109d35b2a3da)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:32:50.345530 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb569a8-491c-46f0-a02b-ac44d54f2c02 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:32:50.361539 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1021/2051] 10.222.0.49 () {60 vars in 1482 bytes} [Wed Jun 15 10:32:50 2022] PUT /identity/v3/projects/02cd089ef0c44ebda9956d2be371fa32/users/184e49aec8cb4c6493c14711a7cdc078/roles/6876e57cc51c448097ab109d35b2a3da => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 10:32:50.372395 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc26535-4936-4c5a-8a22-e440d5140e8d None None] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:32:50.373017 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc26535-4936-4c5a-8a22-e440d5140e8d None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:32:50.373109 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc26535-4936-4c5a-8a22-e440d5140e8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:32:50.404192 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.auth.core [None req-ddc26535-4936-4c5a-8a22-e440d5140e8d None None] MFA Rules not processed for user `184e49aec8cb4c6493c14711a7cdc078`. Rule list: `[]` (Enabled: `True`). {{(pid=80119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:32:50.405425 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-ddc26535-4936-4c5a-8a22-e440d5140e8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:32:50.489282 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1031/2052] 10.222.0.49 () {58 vars in 957 bytes} [Wed Jun 15 10:32:50 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:32:50.502153 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb569a8-491c-46f0-a02b-ac44d54f2c02 admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:32:50.504005 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-8fb569a8-491c-46f0-a02b-ac44d54f2c02 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:32:50.575040 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb569a8-491c-46f0-a02b-ac44d54f2c02 admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:32:50.578896 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07d3bd4-b15d-4ccd-83c5-3d290ae5ecab service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20d1327553404b80aecee87c78fcb52b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '559b3e713f5844b6ae37151d56dd419f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:32:50.579579 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07d3bd4-b15d-4ccd-83c5-3d290ae5ecab service cinder] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:32:50.579873 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07d3bd4-b15d-4ccd-83c5-3d290ae5ecab service cinder] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:32:50.580152 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07d3bd4-b15d-4ccd-83c5-3d290ae5ecab service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:32:50.588902 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc26535-4936-4c5a-8a22-e440d5140e8d None None] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:32:50.589720 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d07d3bd4-b15d-4ccd-83c5-3d290ae5ecab service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:32:50.590305 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-ddc26535-4936-4c5a-8a22-e440d5140e8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:32:50.591207 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d07d3bd4-b15d-4ccd-83c5-3d290ae5ecab service cinder] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:32:50.600511 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1022/2053] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Jun 15 10:32:50 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:32:50.622193 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07d3bd4-b15d-4ccd-83c5-3d290ae5ecab service cinder] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:32:50.663443 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc26535-4936-4c5a-8a22-e440d5140e8d None None] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:32:50.668648 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b896cb36-0659-4a83-9811-73c02e5fdbd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:32:50.669524 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b896cb36-0659-4a83-9811-73c02e5fdbd6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:32:50.669807 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b896cb36-0659-4a83-9811-73c02e5fdbd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:32:50.670084 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b896cb36-0659-4a83-9811-73c02e5fdbd6 admin admin] PATH_INFO: `/v3/users/d18e5e85b8be4225af127b55b6bea9f1` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:32:50.671320 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b896cb36-0659-4a83-9811-73c02e5fdbd6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d18e5e85b8be4225af127b55b6bea9f1)` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:32:50.673255 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b896cb36-0659-4a83-9811-73c02e5fdbd6 admin admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:32:50.686862 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07d3bd4-b15d-4ccd-83c5-3d290ae5ecab service cinder] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:32:50.692474 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81c2901-e4b6-42c9-9f7f-5206d07ade3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:32:50.693375 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81c2901-e4b6-42c9-9f7f-5206d07ade3a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:32:50.693687 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81c2901-e4b6-42c9-9f7f-5206d07ade3a admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:32:50.693961 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81c2901-e4b6-42c9-9f7f-5206d07ade3a admin admin] PATH_INFO: `/v3/users/184e49aec8cb4c6493c14711a7cdc078` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:32:50.695618 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a81c2901-e4b6-42c9-9f7f-5206d07ade3a admin admin] RBAC: Authorizing `identity:delete_user(user_id=184e49aec8cb4c6493c14711a7cdc078)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:32:50.697917 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a81c2901-e4b6-42c9-9f7f-5206d07ade3a admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:32:50.722753 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-b896cb36-0659-4a83-9811-73c02e5fdbd6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd18e5e85b8be4225af127b55b6bea9f1'} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:32:50.739572 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-b896cb36-0659-4a83-9811-73c02e5fdbd6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd18e5e85b8be4225af127b55b6bea9f1'} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:32:50.740493 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-b896cb36-0659-4a83-9811-73c02e5fdbd6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd18e5e85b8be4225af127b55b6bea9f1'} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:32:50.741216 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-b896cb36-0659-4a83-9811-73c02e5fdbd6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd18e5e85b8be4225af127b55b6bea9f1'} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:32:50.749852 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-b896cb36-0659-4a83-9811-73c02e5fdbd6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd18e5e85b8be4225af127b55b6bea9f1'} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:32:50.762881 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-a81c2901-e4b6-42c9-9f7f-5206d07ade3a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '184e49aec8cb4c6493c14711a7cdc078'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:32:50.771212 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1032/2054] 10.222.0.49 () {58 vars in 1223 bytes} [Wed Jun 15 10:32:50 2022] DELETE /identity/v3/users/d18e5e85b8be4225af127b55b6bea9f1 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 10:32:50.771369 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-a81c2901-e4b6-42c9-9f7f-5206d07ade3a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '184e49aec8cb4c6493c14711a7cdc078'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:32:50.771369 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-a81c2901-e4b6-42c9-9f7f-5206d07ade3a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '184e49aec8cb4c6493c14711a7cdc078'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:32:50.771782 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-a81c2901-e4b6-42c9-9f7f-5206d07ade3a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '184e49aec8cb4c6493c14711a7cdc078'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:32:50.774327 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ab2a24-8964-449c-8feb-0e313f7fd14f None None] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:32:50.774696 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ab2a24-8964-449c-8feb-0e313f7fd14f None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:32:50.774984 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ab2a24-8964-449c-8feb-0e313f7fd14f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:32:50.776564 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:32:50.782359 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:32:50.785172 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-a81c2901-e4b6-42c9-9f7f-5206d07ade3a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '184e49aec8cb4c6493c14711a7cdc078'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:32:50.796105 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.auth.core [None req-13ab2a24-8964-449c-8feb-0e313f7fd14f None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:32:50.798576 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-13ab2a24-8964-449c-8feb-0e313f7fd14f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:32:50.807404 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1023/2055] 10.222.0.49 () {58 vars in 1223 bytes} [Wed Jun 15 10:32:50 2022] DELETE /identity/v3/users/184e49aec8cb4c6493c14711a7cdc078 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 10:32:50.811914 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b7ad56-9cba-454a-8a62-6c2c2996dc1c None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:32:50.812400 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b7ad56-9cba-454a-8a62-6c2c2996dc1c None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:32:50.812696 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b7ad56-9cba-454a-8a62-6c2c2996dc1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:32:50.833600 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-88b7ad56-9cba-454a-8a62-6c2c2996dc1c None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:32:50.834934 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-88b7ad56-9cba-454a-8a62-6c2c2996dc1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:32:50.873243 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1033/2056] 10.222.0.49 () {58 vars in 957 bytes} [Wed Jun 15 10:32:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:32:50.885295 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ab2a24-8964-449c-8feb-0e313f7fd14f None None] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:32:50.887007 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-13ab2a24-8964-449c-8feb-0e313f7fd14f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:32:50.908000 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1024/2057] 10.222.0.49 () {58 vars in 957 bytes} [Wed Jun 15 10:32:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 10:32:50.918957 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b7ad56-9cba-454a-8a62-6c2c2996dc1c None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:32:50.919910 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-88b7ad56-9cba-454a-8a62-6c2c2996dc1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:32:50.965745 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ab2a24-8964-449c-8feb-0e313f7fd14f None None] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:32:50.995548 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b7ad56-9cba-454a-8a62-6c2c2996dc1c None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:32:51.002660 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c936ef-f1ab-4ce5-8f66-9992f855e86c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0cb44a17d205431793e9a8791e477a32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '559b3e713f5844b6ae37151d56dd419f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:32:51.004002 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c936ef-f1ab-4ce5-8f66-9992f855e86c service neutron] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:32:51.004409 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c936ef-f1ab-4ce5-8f66-9992f855e86c service neutron] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:32:51.005164 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c936ef-f1ab-4ce5-8f66-9992f855e86c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:32:51.019830 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c936ef-f1ab-4ce5-8f66-9992f855e86c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:32:51.021600 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c936ef-f1ab-4ce5-8f66-9992f855e86c service neutron] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:32:51.036864 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1025/2058] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Jun 15 10:32:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 10:32:51.067505 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c936ef-f1ab-4ce5-8f66-9992f855e86c service neutron] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:32:51.069122 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-a3c936ef-f1ab-4ce5-8f66-9992f855e86c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:32:51.073303 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56027eb6-06c2-4e4d-8cd3-d1388f8c53ab service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0cb44a17d205431793e9a8791e477a32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '559b3e713f5844b6ae37151d56dd419f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:32:51.074005 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56027eb6-06c2-4e4d-8cd3-d1388f8c53ab service neutron] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:32:51.074297 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56027eb6-06c2-4e4d-8cd3-d1388f8c53ab service neutron] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:32:51.074669 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56027eb6-06c2-4e4d-8cd3-d1388f8c53ab service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:32:51.089530 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c936ef-f1ab-4ce5-8f66-9992f855e86c service neutron] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:32:51.103398 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56027eb6-06c2-4e4d-8cd3-d1388f8c53ab service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:32:51.104537 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4d08f2-2506-4671-8bb9-944832937c7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:32:51.104897 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56027eb6-06c2-4e4d-8cd3-d1388f8c53ab service neutron] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:32:51.106438 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4d08f2-2506-4671-8bb9-944832937c7d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:32:51.106764 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4d08f2-2506-4671-8bb9-944832937c7d admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:32:51.107065 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4d08f2-2506-4671-8bb9-944832937c7d admin admin] PATH_INFO: `/v3/users/320d3accb5ae426387f8f46928473ab0` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:32:51.108446 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c4d08f2-2506-4671-8bb9-944832937c7d admin admin] RBAC: Authorizing `identity:delete_user(user_id=320d3accb5ae426387f8f46928473ab0)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:32:51.111644 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c4d08f2-2506-4671-8bb9-944832937c7d admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:32:51.122447 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1034/2059] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Jun 15 10:32:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 238 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:32:51.190492 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-2c4d08f2-2506-4671-8bb9-944832937c7d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '320d3accb5ae426387f8f46928473ab0'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:32:51.201052 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-2c4d08f2-2506-4671-8bb9-944832937c7d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '320d3accb5ae426387f8f46928473ab0'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:32:51.201821 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-2c4d08f2-2506-4671-8bb9-944832937c7d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '320d3accb5ae426387f8f46928473ab0'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:32:51.203041 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-2c4d08f2-2506-4671-8bb9-944832937c7d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '320d3accb5ae426387f8f46928473ab0'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:32:51.223589 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-2c4d08f2-2506-4671-8bb9-944832937c7d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '320d3accb5ae426387f8f46928473ab0'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:32:51.234934 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:DELETE=12,keystone:SELECT=87,keystone:INSERT=1 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:32:51.251022 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1026/2060] 10.222.0.49 () {58 vars in 1223 bytes} [Wed Jun 15 10:32:51 2022] DELETE /identity/v3/users/320d3accb5ae426387f8f46928473ab0 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 10:32:51.257155 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5608119-2d9d-4a92-8360-88dd378aa682 None None] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:32:51.257742 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5608119-2d9d-4a92-8360-88dd378aa682 None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:32:51.257742 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5608119-2d9d-4a92-8360-88dd378aa682 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:32:51.278330 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.auth.core [None req-b5608119-2d9d-4a92-8360-88dd378aa682 None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:32:51.280582 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-b5608119-2d9d-4a92-8360-88dd378aa682 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:32:51.380140 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1035/2061] 10.222.0.49 () {58 vars in 957 bytes} [Wed Jun 15 10:32:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:32:51.394410 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4d08f2-2506-4671-8bb9-944832937c7d admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:32:51.396021 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-2c4d08f2-2506-4671-8bb9-944832937c7d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:32:51.473317 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4d08f2-2506-4671-8bb9-944832937c7d admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:32:51.477509 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec939666-612a-4039-9b10-c4b883c276a8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0cb44a17d205431793e9a8791e477a32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '559b3e713f5844b6ae37151d56dd419f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:32:51.478205 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec939666-612a-4039-9b10-c4b883c276a8 service neutron] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:32:51.478454 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec939666-612a-4039-9b10-c4b883c276a8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:32:51.478734 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec939666-612a-4039-9b10-c4b883c276a8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:32:51.490638 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec939666-612a-4039-9b10-c4b883c276a8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:32:51.495395 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec939666-612a-4039-9b10-c4b883c276a8 service neutron] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:32:51.509899 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1027/2062] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Jun 15 10:32:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:32:51.671239 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5608119-2d9d-4a92-8360-88dd378aa682 None None] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:32:51.672719 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-b5608119-2d9d-4a92-8360-88dd378aa682 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:32:51.680898 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=100 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:32:51.686293 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec939666-612a-4039-9b10-c4b883c276a8 service neutron] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:32:51.688032 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-ec939666-612a-4039-9b10-c4b883c276a8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:32:51.693797 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5608119-2d9d-4a92-8360-88dd378aa682 None None] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:32:51.704398 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec939666-612a-4039-9b10-c4b883c276a8 service neutron] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:32:51.709107 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa35e4a-d7d3-4537-89ff-10567ccda092 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:32:51.709536 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2130c7e1-b705-41ce-9ade-bf2945b39ba3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:32:51.709973 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa35e4a-d7d3-4537-89ff-10567ccda092 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:32:51.710258 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa35e4a-d7d3-4537-89ff-10567ccda092 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:32:51.710432 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2130c7e1-b705-41ce-9ade-bf2945b39ba3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:32:51.710527 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa35e4a-d7d3-4537-89ff-10567ccda092 admin admin] PATH_INFO: `/v3/projects/5c0b3805b6d74092ac6710bb6b00cb1e` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:32:51.710701 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2130c7e1-b705-41ce-9ade-bf2945b39ba3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:32:51.710983 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2130c7e1-b705-41ce-9ade-bf2945b39ba3 admin admin] PATH_INFO: `/v3/projects/02cd089ef0c44ebda9956d2be371fa32` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:32:51.711987 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa35e4a-d7d3-4537-89ff-10567ccda092 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5c0b3805b6d74092ac6710bb6b00cb1e)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:32:51.713751 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa35e4a-d7d3-4537-89ff-10567ccda092 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:32:51.714672 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2130c7e1-b705-41ce-9ade-bf2945b39ba3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=02cd089ef0c44ebda9956d2be371fa32)` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:32:51.716651 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2130c7e1-b705-41ce-9ade-bf2945b39ba3 admin admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:32:51.781875 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-9fa35e4a-d7d3-4537-89ff-10567ccda092 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5c0b3805b6d74092ac6710bb6b00cb1e', 'request_id': 'req-9fa35e4a-d7d3-4537-89ff-10567ccda092'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:32:51.799970 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-9fa35e4a-d7d3-4537-89ff-10567ccda092 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5c0b3805b6d74092ac6710bb6b00cb1e', 'request_id': 'req-9fa35e4a-d7d3-4537-89ff-10567ccda092'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:32:51.821482 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-9fa35e4a-d7d3-4537-89ff-10567ccda092 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5c0b3805b6d74092ac6710bb6b00cb1e', 'request_id': 'req-9fa35e4a-d7d3-4537-89ff-10567ccda092'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:32:51.837148 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-9fa35e4a-d7d3-4537-89ff-10567ccda092 admin admin] The token cache is being invalidate because project 5c0b3805b6d74092ac6710bb6b00cb1e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80120) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 10:32:51.837520 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-9fa35e4a-d7d3-4537-89ff-10567ccda092 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:32:51.838200 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-9fa35e4a-d7d3-4537-89ff-10567ccda092 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:32:51.839858 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1028/2063] 10.222.0.49 () {58 vars in 1232 bytes} [Wed Jun 15 10:32:51 2022] DELETE /identity/v3/projects/5c0b3805b6d74092ac6710bb6b00cb1e => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:32:51.843878 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-2130c7e1-b705-41ce-9ade-bf2945b39ba3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '02cd089ef0c44ebda9956d2be371fa32', 'request_id': 'req-2130c7e1-b705-41ce-9ade-bf2945b39ba3'} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:32:51.844062 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa35e4a-d7d3-4537-89ff-10567ccda092 admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:32:51.845905 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-9fa35e4a-d7d3-4537-89ff-10567ccda092 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:32:51.855493 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-2130c7e1-b705-41ce-9ade-bf2945b39ba3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '02cd089ef0c44ebda9956d2be371fa32', 'request_id': 'req-2130c7e1-b705-41ce-9ade-bf2945b39ba3'} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:32:51.867178 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-2130c7e1-b705-41ce-9ade-bf2945b39ba3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '02cd089ef0c44ebda9956d2be371fa32', 'request_id': 'req-2130c7e1-b705-41ce-9ade-bf2945b39ba3'} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:32:51.885891 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-2130c7e1-b705-41ce-9ade-bf2945b39ba3 admin admin] The token cache is being invalidate because project 02cd089ef0c44ebda9956d2be371fa32 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80119) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 10:32:51.886191 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-2130c7e1-b705-41ce-9ade-bf2945b39ba3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:32:51.887004 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-2130c7e1-b705-41ce-9ade-bf2945b39ba3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:32:51.891888 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1036/2064] 10.222.0.49 () {58 vars in 1232 bytes} [Wed Jun 15 10:32:51 2022] DELETE /identity/v3/projects/02cd089ef0c44ebda9956d2be371fa32 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:32:51.896697 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284d5098-ec00-47b9-a242-941b468a3c88 None None] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:32:51.901296 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284d5098-ec00-47b9-a242-941b468a3c88 None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:32:51.901654 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284d5098-ec00-47b9-a242-941b468a3c88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:32:51.929871 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.auth.core [None req-284d5098-ec00-47b9-a242-941b468a3c88 None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:32:51.940738 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-284d5098-ec00-47b9-a242-941b468a3c88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:32:51.962259 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa35e4a-d7d3-4537-89ff-10567ccda092 admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:32:51.994594 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:32:52.023253 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1037/2065] 10.222.0.49 () {58 vars in 957 bytes} [Wed Jun 15 10:32:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:32:52.029832 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284d5098-ec00-47b9-a242-941b468a3c88 None None] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:32:52.036489 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-284d5098-ec00-47b9-a242-941b468a3c88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:32:52.044446 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c8aee7-6fc2-492f-90fb-8ac85c55568f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:32:52.045631 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c8aee7-6fc2-492f-90fb-8ac85c55568f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:32:52.045985 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c8aee7-6fc2-492f-90fb-8ac85c55568f admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:32:52.046449 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c8aee7-6fc2-492f-90fb-8ac85c55568f admin admin] PATH_INFO: `/v3/users/4eee6512e3cf47bf989a0b9a8bebb088` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:32:52.047882 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c8aee7-6fc2-492f-90fb-8ac85c55568f admin admin] RBAC: Authorizing `identity:delete_user(user_id=4eee6512e3cf47bf989a0b9a8bebb088)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:32:52.050347 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c8aee7-6fc2-492f-90fb-8ac85c55568f admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:32:52.050698 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284d5098-ec00-47b9-a242-941b468a3c88 None None] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:32:52.055657 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16422c86-2607-4b41-8035-0323584a2cfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:32:52.056522 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16422c86-2607-4b41-8035-0323584a2cfc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:32:52.056803 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16422c86-2607-4b41-8035-0323584a2cfc admin admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:32:52.057072 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16422c86-2607-4b41-8035-0323584a2cfc admin admin] PATH_INFO: `/v3/projects/3e00222d3870456187378cdc6ccb499a` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:32:52.058240 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16422c86-2607-4b41-8035-0323584a2cfc admin admin] RBAC: Authorizing `identity:delete_project(project_id=3e00222d3870456187378cdc6ccb499a)` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:32:52.060306 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16422c86-2607-4b41-8035-0323584a2cfc admin admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:32:52.104138 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-01c8aee7-6fc2-492f-90fb-8ac85c55568f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4eee6512e3cf47bf989a0b9a8bebb088'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:32:52.111837 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-16422c86-2607-4b41-8035-0323584a2cfc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3e00222d3870456187378cdc6ccb499a', 'request_id': 'req-16422c86-2607-4b41-8035-0323584a2cfc'} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:32:52.113888 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-01c8aee7-6fc2-492f-90fb-8ac85c55568f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4eee6512e3cf47bf989a0b9a8bebb088'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:32:52.114654 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-01c8aee7-6fc2-492f-90fb-8ac85c55568f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4eee6512e3cf47bf989a0b9a8bebb088'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:32:52.115412 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-01c8aee7-6fc2-492f-90fb-8ac85c55568f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4eee6512e3cf47bf989a0b9a8bebb088'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:32:52.120131 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-16422c86-2607-4b41-8035-0323584a2cfc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3e00222d3870456187378cdc6ccb499a', 'request_id': 'req-16422c86-2607-4b41-8035-0323584a2cfc'} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:32:52.123460 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-01c8aee7-6fc2-492f-90fb-8ac85c55568f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4eee6512e3cf47bf989a0b9a8bebb088'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:32:52.126359 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-16422c86-2607-4b41-8035-0323584a2cfc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3e00222d3870456187378cdc6ccb499a', 'request_id': 'req-16422c86-2607-4b41-8035-0323584a2cfc'} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:32:52.137911 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-16422c86-2607-4b41-8035-0323584a2cfc admin admin] The token cache is being invalidate because project 3e00222d3870456187378cdc6ccb499a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80119) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 10:32:52.138303 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-16422c86-2607-4b41-8035-0323584a2cfc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:32:52.139047 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-16422c86-2607-4b41-8035-0323584a2cfc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:32:52.140835 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1038/2066] 10.222.0.49 () {58 vars in 1232 bytes} [Wed Jun 15 10:32:52 2022] DELETE /identity/v3/projects/3e00222d3870456187378cdc6ccb499a => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:32:52.142101 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1029/2067] 10.222.0.49 () {58 vars in 1223 bytes} [Wed Jun 15 10:32:51 2022] DELETE /identity/v3/users/4eee6512e3cf47bf989a0b9a8bebb088 => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:32:52.142355 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d593b527-8d1a-484d-81fe-56e14cae0688 None None] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:32:52.142676 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d593b527-8d1a-484d-81fe-56e14cae0688 None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:32:52.143009 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d593b527-8d1a-484d-81fe-56e14cae0688 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:32:52.146565 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6f49d3-6290-47fb-ab36-a4e89576c40e None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:32:52.146945 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6f49d3-6290-47fb-ab36-a4e89576c40e None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:32:52.147265 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6f49d3-6290-47fb-ab36-a4e89576c40e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:32:52.163648 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.auth.core [None req-d593b527-8d1a-484d-81fe-56e14cae0688 None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:32:52.169632 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-d593b527-8d1a-484d-81fe-56e14cae0688 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:32:52.169812 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-9e6f49d3-6290-47fb-ab36-a4e89576c40e None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:32:52.169812 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-9e6f49d3-6290-47fb-ab36-a4e89576c40e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:32:52.175748 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:32:52.253124 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1030/2068] 10.222.0.49 () {58 vars in 957 bytes} [Wed Jun 15 10:32:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:32:52.260801 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1039/2069] 10.222.0.49 () {58 vars in 957 bytes} [Wed Jun 15 10:32:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:32:52.263861 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c2f786-cece-4a06-8d5d-150dccae9e7f None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:32:52.264525 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c2f786-cece-4a06-8d5d-150dccae9e7f None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:32:52.264931 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c2f786-cece-4a06-8d5d-150dccae9e7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:32:52.265108 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d593b527-8d1a-484d-81fe-56e14cae0688 None None] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:32:52.275858 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d593b527-8d1a-484d-81fe-56e14cae0688 None None] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:32:52.282004 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8efdbc-cf8d-4de5-b63a-4e9e02d567aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:32:52.282689 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8efdbc-cf8d-4de5-b63a-4e9e02d567aa admin admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:32:52.282975 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8efdbc-cf8d-4de5-b63a-4e9e02d567aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:32:52.283299 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8efdbc-cf8d-4de5-b63a-4e9e02d567aa admin admin] PATH_INFO: `/v3/domains` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:32:52.284085 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8efdbc-cf8d-4de5-b63a-4e9e02d567aa admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:32:52.284537 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8efdbc-cf8d-4de5-b63a-4e9e02d567aa admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:32:52.286218 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8efdbc-cf8d-4de5-b63a-4e9e02d567aa admin admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:32:52.291258 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-00c2f786-cece-4a06-8d5d-150dccae9e7f None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:32:52.293076 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-00c2f786-cece-4a06-8d5d-150dccae9e7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:32:52.303273 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1040/2070] 10.222.0.49 () {58 vars in 1152 bytes} [Wed Jun 15 10:32:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 10:32:52.306776 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1031/2071] 10.222.0.49 () {58 vars in 957 bytes} [Wed Jun 15 10:32:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 10:32:52.309964 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8efdbc-cf8d-4de5-b63a-4e9e02d567aa admin admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:32:52.311259 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c2f786-cece-4a06-8d5d-150dccae9e7f None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:32:52.322604 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c2f786-cece-4a06-8d5d-150dccae9e7f None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:32:52.323107 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8efdbc-cf8d-4de5-b63a-4e9e02d567aa admin admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:32:52.337937 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c635c61-47ef-4a3d-af7f-554f6c3bf6c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:32:52.338632 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c635c61-47ef-4a3d-af7f-554f6c3bf6c3 admin admin] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:32:52.338926 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c635c61-47ef-4a3d-af7f-554f6c3bf6c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:32:52.339231 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c635c61-47ef-4a3d-af7f-554f6c3bf6c3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:32:52.343565 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c635c61-47ef-4a3d-af7f-554f6c3bf6c3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:32:52.346135 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c635c61-47ef-4a3d-af7f-554f6c3bf6c3 admin admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:32:52.348321 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18214ff2-fcea-4cf8-8b1d-d2f841497c33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:32:52.349985 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18214ff2-fcea-4cf8-8b1d-d2f841497c33 admin admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:32:52.350391 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18214ff2-fcea-4cf8-8b1d-d2f841497c33 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:32:52.351375 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18214ff2-fcea-4cf8-8b1d-d2f841497c33 admin admin] PATH_INFO: `/v3/domains` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:32:52.353853 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18214ff2-fcea-4cf8-8b1d-d2f841497c33 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:32:52.354807 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18214ff2-fcea-4cf8-8b1d-d2f841497c33 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:32:52.358149 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18214ff2-fcea-4cf8-8b1d-d2f841497c33 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:32:52.378049 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1032/2072] 10.222.0.49 () {58 vars in 1152 bytes} [Wed Jun 15 10:32:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:32:52.384754 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18214ff2-fcea-4cf8-8b1d-d2f841497c33 admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:32:52.398457 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18214ff2-fcea-4cf8-8b1d-d2f841497c33 admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:32:52.405263 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70e4260-403d-4804-873b-55ebabf44b09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:32:52.408847 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1041/2073] 10.222.0.49 () {60 vars in 1152 bytes} [Wed Jun 15 10:32:52 2022] POST /identity/v3/projects => generated 361 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 10:32:52.409445 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70e4260-403d-4804-873b-55ebabf44b09 admin admin] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:32:52.409445 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70e4260-403d-4804-873b-55ebabf44b09 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:32:52.409445 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70e4260-403d-4804-873b-55ebabf44b09 admin admin] PATH_INFO: `/v3/projects` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:32:52.409445 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b70e4260-403d-4804-873b-55ebabf44b09 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:32:52.410000 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c635c61-47ef-4a3d-af7f-554f6c3bf6c3 admin admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:32:52.411708 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b70e4260-403d-4804-873b-55ebabf44b09 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:32:52.462633 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1033/2074] 10.222.0.49 () {60 vars in 1152 bytes} [Wed Jun 15 10:32:52 2022] POST /identity/v3/projects => generated 383 bytes in 76 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 10:32:52.464404 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70e4260-403d-4804-873b-55ebabf44b09 admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:32:52.473871 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:32:52.525441 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c635c61-47ef-4a3d-af7f-554f6c3bf6c3 admin admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:32:52.605819 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e91b5d-4d91-4080-880d-3ef693e3b724 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:32:52.606615 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e91b5d-4d91-4080-880d-3ef693e3b724 admin admin] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:32:52.606980 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e91b5d-4d91-4080-880d-3ef693e3b724 admin admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:32:52.607370 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e91b5d-4d91-4080-880d-3ef693e3b724 admin admin] PATH_INFO: `/v3/users` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:32:52.608170 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e91b5d-4d91-4080-880d-3ef693e3b724 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:32:52.610327 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e91b5d-4d91-4080-880d-3ef693e3b724 admin admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:32:52.612143 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70e4260-403d-4804-873b-55ebabf44b09 admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:32:52.617870 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ce60d4-09e3-4940-b625-d8dc5e953468 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:32:52.619203 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ce60d4-09e3-4940-b625-d8dc5e953468 admin admin] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:32:52.619543 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ce60d4-09e3-4940-b625-d8dc5e953468 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:32:52.619799 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ce60d4-09e3-4940-b625-d8dc5e953468 admin admin] PATH_INFO: `/v3/users` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:32:52.620644 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ce60d4-09e3-4940-b625-d8dc5e953468 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:32:52.622735 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ce60d4-09e3-4940-b625-d8dc5e953468 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:32:52.630788 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1042/2075] 10.222.0.49 () {60 vars in 1143 bytes} [Wed Jun 15 10:32:52 2022] POST /identity/v3/users => generated 281 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 10:32:52.635051 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e91b5d-4d91-4080-880d-3ef693e3b724 admin admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:32:52.637056 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-76e91b5d-4d91-4080-880d-3ef693e3b724 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:32:52.645541 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1034/2076] 10.222.0.49 () {60 vars in 1143 bytes} [Wed Jun 15 10:32:52 2022] POST /identity/v3/users => generated 292 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 10:32:52.649899 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ce60d4-09e3-4940-b625-d8dc5e953468 admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:32:52.651358 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e91b5d-4d91-4080-880d-3ef693e3b724 admin admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:32:52.658166 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1bd2c2-59bf-4406-9b2c-a259280291d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:32:52.659357 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1bd2c2-59bf-4406-9b2c-a259280291d7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:32:52.659680 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1bd2c2-59bf-4406-9b2c-a259280291d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:32:52.659968 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1bd2c2-59bf-4406-9b2c-a259280291d7 admin admin] PATH_INFO: `/v3/projects/974ed5ca2296425fbc2cea2aee4c3ca3` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:32:52.661526 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d1bd2c2-59bf-4406-9b2c-a259280291d7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=974ed5ca2296425fbc2cea2aee4c3ca3)` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:32:52.663691 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ce60d4-09e3-4940-b625-d8dc5e953468 admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:32:52.663824 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d1bd2c2-59bf-4406-9b2c-a259280291d7 admin admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:32:52.670325 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93a414f-7e10-4ab2-be85-23a8962ad97f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:32:52.671187 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93a414f-7e10-4ab2-be85-23a8962ad97f admin admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:32:52.671499 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:32:52.671640 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93a414f-7e10-4ab2-be85-23a8962ad97f admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:32:52.672242 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93a414f-7e10-4ab2-be85-23a8962ad97f admin admin] PATH_INFO: `/v3/roles` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:32:52.672870 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b93a414f-7e10-4ab2-be85-23a8962ad97f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:32:52.674588 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b93a414f-7e10-4ab2-be85-23a8962ad97f admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:32:52.686808 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1035/2077] 10.222.0.49 () {58 vars in 1121 bytes} [Wed Jun 15 10:32:52 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 10:32:52.690820 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93a414f-7e10-4ab2-be85-23a8962ad97f admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:32:52.701928 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93a414f-7e10-4ab2-be85-23a8962ad97f admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:32:52.715548 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1db4bf-ec5c-4d4b-950e-33ba121fc169 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:32:52.715548 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1db4bf-ec5c-4d4b-950e-33ba121fc169 admin admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:32:52.715548 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1db4bf-ec5c-4d4b-950e-33ba121fc169 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:32:52.715548 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1db4bf-ec5c-4d4b-950e-33ba121fc169 admin admin] PATH_INFO: `/v3/roles` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:32:52.715927 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac1db4bf-ec5c-4d4b-950e-33ba121fc169 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:32:52.715927 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac1db4bf-ec5c-4d4b-950e-33ba121fc169 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:32:52.719733 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-6d1bd2c2-59bf-4406-9b2c-a259280291d7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '974ed5ca2296425fbc2cea2aee4c3ca3', 'request_id': 'req-6d1bd2c2-59bf-4406-9b2c-a259280291d7'} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:32:52.722583 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1036/2078] 10.222.0.49 () {58 vars in 1121 bytes} [Wed Jun 15 10:32:52 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 10:32:52.723427 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1db4bf-ec5c-4d4b-950e-33ba121fc169 admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:32:52.728626 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-6d1bd2c2-59bf-4406-9b2c-a259280291d7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '974ed5ca2296425fbc2cea2aee4c3ca3', 'request_id': 'req-6d1bd2c2-59bf-4406-9b2c-a259280291d7'} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:32:52.734744 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-6d1bd2c2-59bf-4406-9b2c-a259280291d7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '974ed5ca2296425fbc2cea2aee4c3ca3', 'request_id': 'req-6d1bd2c2-59bf-4406-9b2c-a259280291d7'} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:32:52.734909 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1db4bf-ec5c-4d4b-950e-33ba121fc169 admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:32:52.747967 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-6d1bd2c2-59bf-4406-9b2c-a259280291d7 admin admin] The token cache is being invalidate because project 974ed5ca2296425fbc2cea2aee4c3ca3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80119) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 10:32:52.748161 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-6d1bd2c2-59bf-4406-9b2c-a259280291d7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:32:52.748848 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-6d1bd2c2-59bf-4406-9b2c-a259280291d7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:32:52.750544 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1043/2079] 10.222.0.49 () {58 vars in 1232 bytes} [Wed Jun 15 10:32:52 2022] DELETE /identity/v3/projects/974ed5ca2296425fbc2cea2aee4c3ca3 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:32:52.754073 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1bd2c2-59bf-4406-9b2c-a259280291d7 admin admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:32:52.755506 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-6d1bd2c2-59bf-4406-9b2c-a259280291d7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:32:52.795730 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca145f0-7b6a-4abb-a4f6-e687953fa98f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:32:52.796432 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca145f0-7b6a-4abb-a4f6-e687953fa98f admin admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:32:52.796716 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca145f0-7b6a-4abb-a4f6-e687953fa98f admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:32:52.796990 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca145f0-7b6a-4abb-a4f6-e687953fa98f admin admin] PATH_INFO: `/v3/roles` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:32:52.797792 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca145f0-7b6a-4abb-a4f6-e687953fa98f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:32:52.799511 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca145f0-7b6a-4abb-a4f6-e687953fa98f admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:32:52.810024 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1037/2080] 10.222.0.49 () {58 vars in 1121 bytes} [Wed Jun 15 10:32:52 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 10:32:52.811302 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96429796-ca93-4053-a272-d84b41b4cec1 None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:32:52.811590 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96429796-ca93-4053-a272-d84b41b4cec1 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:32:52.811867 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96429796-ca93-4053-a272-d84b41b4cec1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:32:52.833750 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-96429796-ca93-4053-a272-d84b41b4cec1 None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:32:52.835117 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-96429796-ca93-4053-a272-d84b41b4cec1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:32:52.837509 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1bd2c2-59bf-4406-9b2c-a259280291d7 admin admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:32:52.842229 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a1377e-19bc-4aa9-adcc-608ccde2dfff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:32:52.842969 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a1377e-19bc-4aa9-adcc-608ccde2dfff admin admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:32:52.843373 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a1377e-19bc-4aa9-adcc-608ccde2dfff admin admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:32:52.843696 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1038/2081] 10.222.0.49 () {58 vars in 957 bytes} [Wed Jun 15 10:32:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:32:52.849981 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a1377e-19bc-4aa9-adcc-608ccde2dfff admin admin] PATH_INFO: `/v3/roles` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:32:52.850111 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96429796-ca93-4053-a272-d84b41b4cec1 None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:32:52.850227 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a1377e-19bc-4aa9-adcc-608ccde2dfff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:32:52.850321 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-96429796-ca93-4053-a272-d84b41b4cec1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:32:52.850408 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a1377e-19bc-4aa9-adcc-608ccde2dfff admin admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:32:52.860214 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1044/2082] 10.222.0.49 () {58 vars in 1121 bytes} [Wed Jun 15 10:32:52 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 10:32:52.860375 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96429796-ca93-4053-a272-d84b41b4cec1 None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:32:52.860492 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b4dc5c-51aa-4453-9614-99443a88d825 None None] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:32:52.860492 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b4dc5c-51aa-4453-9614-99443a88d825 None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:32:52.860492 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b4dc5c-51aa-4453-9614-99443a88d825 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:32:52.863248 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41169ef0-4db5-4bb8-bdd4-3a94351730bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:32:52.864130 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41169ef0-4db5-4bb8-bdd4-3a94351730bc admin admin] REQUEST_METHOD: `PUT` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:32:52.864525 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41169ef0-4db5-4bb8-bdd4-3a94351730bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:32:52.867198 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41169ef0-4db5-4bb8-bdd4-3a94351730bc admin admin] PATH_INFO: `/v3/projects/f8036511910d43eab14c26921b0d730c/users/25b88b19c2784a7b987fa4904c951c75/roles/6876e57cc51c448097ab109d35b2a3da` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:32:52.877147 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.auth.core [None req-23b4dc5c-51aa-4453-9614-99443a88d825 None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:32:52.878463 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-23b4dc5c-51aa-4453-9614-99443a88d825 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:32:52.881998 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41169ef0-4db5-4bb8-bdd4-3a94351730bc admin admin] RBAC: Authorizing `identity:create_grant(project_id=f8036511910d43eab14c26921b0d730c, user_id=25b88b19c2784a7b987fa4904c951c75, role_id=6876e57cc51c448097ab109d35b2a3da)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:32:52.884738 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41169ef0-4db5-4bb8-bdd4-3a94351730bc admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:32:52.886906 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1045/2083] 10.222.0.49 () {58 vars in 957 bytes} [Wed Jun 15 10:32:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:32:52.887849 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b4dc5c-51aa-4453-9614-99443a88d825 None None] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:32:52.900062 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b4dc5c-51aa-4453-9614-99443a88d825 None None] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:32:52.909743 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1039/2084] 10.222.0.49 () {60 vars in 1482 bytes} [Wed Jun 15 10:32:52 2022] PUT /identity/v3/projects/f8036511910d43eab14c26921b0d730c/users/25b88b19c2784a7b987fa4904c951c75/roles/6876e57cc51c448097ab109d35b2a3da => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:32:52.911012 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41169ef0-4db5-4bb8-bdd4-3a94351730bc admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:32:52.952615 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:32:52.970317 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a721587-7459-4d0e-b1ea-8d297ab155c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:32:52.971266 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a721587-7459-4d0e-b1ea-8d297ab155c5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:32:52.971555 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a721587-7459-4d0e-b1ea-8d297ab155c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:32:52.971836 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a721587-7459-4d0e-b1ea-8d297ab155c5 admin admin] PATH_INFO: `/v3/projects/722b2275169048cc8263fd39ac8c0959/users/139f35f69f4940c69700535d4cd53d50/roles/6876e57cc51c448097ab109d35b2a3da` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:32:52.980889 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41169ef0-4db5-4bb8-bdd4-3a94351730bc admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:32:52.985519 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee57fad-67fa-46ea-99cc-80652289645f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:32:52.986176 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee57fad-67fa-46ea-99cc-80652289645f admin admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:32:52.986479 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee57fad-67fa-46ea-99cc-80652289645f admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:32:52.986732 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee57fad-67fa-46ea-99cc-80652289645f admin admin] PATH_INFO: `/v3/domains` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:32:52.987451 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a721587-7459-4d0e-b1ea-8d297ab155c5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=722b2275169048cc8263fd39ac8c0959, user_id=139f35f69f4940c69700535d4cd53d50, role_id=6876e57cc51c448097ab109d35b2a3da)` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:32:52.987611 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee57fad-67fa-46ea-99cc-80652289645f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:32:52.987875 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee57fad-67fa-46ea-99cc-80652289645f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:32:52.989466 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee57fad-67fa-46ea-99cc-80652289645f admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:32:52.990419 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a721587-7459-4d0e-b1ea-8d297ab155c5 admin admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:32:53.002096 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1040/2085] 10.222.0.49 () {58 vars in 1152 bytes} [Wed Jun 15 10:32:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:32:53.003309 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21d2e6f-638a-413d-b427-0e56a573d896 None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:32:53.003597 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21d2e6f-638a-413d-b427-0e56a573d896 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:32:53.003874 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21d2e6f-638a-413d-b427-0e56a573d896 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:32:53.007836 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1046/2086] 10.222.0.49 () {60 vars in 1482 bytes} [Wed Jun 15 10:32:52 2022] PUT /identity/v3/projects/722b2275169048cc8263fd39ac8c0959/users/139f35f69f4940c69700535d4cd53d50/roles/6876e57cc51c448097ab109d35b2a3da => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:32:53.008658 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a721587-7459-4d0e-b1ea-8d297ab155c5 admin admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:32:53.022284 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:32:53.031896 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-d21d2e6f-638a-413d-b427-0e56a573d896 None None] MFA Rules not processed for user `25b88b19c2784a7b987fa4904c951c75`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:32:53.033348 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-d21d2e6f-638a-413d-b427-0e56a573d896 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:32:53.081546 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a721587-7459-4d0e-b1ea-8d297ab155c5 admin admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:32:53.086223 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9443b3b-bbc0-4919-9c3a-09bd2b99667e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:32:53.086878 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9443b3b-bbc0-4919-9c3a-09bd2b99667e admin admin] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:32:53.087302 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9443b3b-bbc0-4919-9c3a-09bd2b99667e admin admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:32:53.087586 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9443b3b-bbc0-4919-9c3a-09bd2b99667e admin admin] PATH_INFO: `/v3/projects` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:32:53.088290 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9443b3b-bbc0-4919-9c3a-09bd2b99667e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:32:53.090422 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9443b3b-bbc0-4919-9c3a-09bd2b99667e admin admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:32:53.099714 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1041/2087] 10.222.0.49 () {58 vars in 957 bytes} [Wed Jun 15 10:32:53 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:32:53.100849 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c3dcdc-cb46-4760-b482-6de3143bc5c1 None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:32:53.101141 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c3dcdc-cb46-4760-b482-6de3143bc5c1 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:32:53.101424 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c3dcdc-cb46-4760-b482-6de3143bc5c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:32:53.132389 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1047/2088] 10.222.0.49 () {60 vars in 1152 bytes} [Wed Jun 15 10:32:53 2022] POST /identity/v3/projects => generated 367 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 10:32:53.134052 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9443b3b-bbc0-4919-9c3a-09bd2b99667e admin admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:32:53.136385 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-02c3dcdc-cb46-4760-b482-6de3143bc5c1 None None] MFA Rules not processed for user `139f35f69f4940c69700535d4cd53d50`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:32:53.137698 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-02c3dcdc-cb46-4760-b482-6de3143bc5c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:32:53.199899 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9443b3b-bbc0-4919-9c3a-09bd2b99667e admin admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:32:53.203314 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1042/2089] 10.222.0.49 () {58 vars in 957 bytes} [Wed Jun 15 10:32:53 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:32:53.204774 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3bf71bc-a785-4bd5-a6d6-336299f1c23f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:32:53.205518 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bf71bc-a785-4bd5-a6d6-336299f1c23f admin admin] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:32:53.205877 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bf71bc-a785-4bd5-a6d6-336299f1c23f admin admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:32:53.206229 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bf71bc-a785-4bd5-a6d6-336299f1c23f admin admin] PATH_INFO: `/v3/users` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:32:53.206985 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3bf71bc-a785-4bd5-a6d6-336299f1c23f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:32:53.208189 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c3dcdc-cb46-4760-b482-6de3143bc5c1 None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:32:53.212156 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-02c3dcdc-cb46-4760-b482-6de3143bc5c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:32:53.213581 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3bf71bc-a785-4bd5-a6d6-336299f1c23f admin admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:32:53.234794 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1048/2090] 10.222.0.49 () {60 vars in 1143 bytes} [Wed Jun 15 10:32:53 2022] POST /identity/v3/users => generated 284 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 10:32:53.235830 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3bf71bc-a785-4bd5-a6d6-336299f1c23f admin admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:32:53.250717 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3bf71bc-a785-4bd5-a6d6-336299f1c23f admin admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:32:53.257563 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171979fd-31e1-48a1-9674-ff3ad353383a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:32:53.257563 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171979fd-31e1-48a1-9674-ff3ad353383a admin admin] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:32:53.257563 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171979fd-31e1-48a1-9674-ff3ad353383a admin admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:32:53.257563 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171979fd-31e1-48a1-9674-ff3ad353383a admin admin] PATH_INFO: `/v3/projects` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:32:53.257935 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171979fd-31e1-48a1-9674-ff3ad353383a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:32:53.259414 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171979fd-31e1-48a1-9674-ff3ad353383a admin admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:32:53.274719 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:32:53.279597 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c3dcdc-cb46-4760-b482-6de3143bc5c1 None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:32:53.284207 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39621dab-0ff0-4cf8-86f8-ace909fac62e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20d1327553404b80aecee87c78fcb52b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '559b3e713f5844b6ae37151d56dd419f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:32:53.284864 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39621dab-0ff0-4cf8-86f8-ace909fac62e service cinder] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:32:53.285149 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39621dab-0ff0-4cf8-86f8-ace909fac62e service cinder] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:32:53.285425 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39621dab-0ff0-4cf8-86f8-ace909fac62e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:32:53.287515 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1049/2091] 10.222.0.49 () {60 vars in 1152 bytes} [Wed Jun 15 10:32:53 2022] POST /identity/v3/projects => generated 383 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 10:32:53.288342 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171979fd-31e1-48a1-9674-ff3ad353383a admin admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:32:53.353228 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39621dab-0ff0-4cf8-86f8-ace909fac62e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:32:53.354691 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39621dab-0ff0-4cf8-86f8-ace909fac62e service cinder] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:32:53.360388 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171979fd-31e1-48a1-9674-ff3ad353383a admin admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:32:53.363798 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1043/2092] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Jun 15 10:32:53 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:32:53.364776 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39621dab-0ff0-4cf8-86f8-ace909fac62e service cinder] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:32:53.365288 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686b35c3-007e-4690-887c-bfb592bab3a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:32:53.366032 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686b35c3-007e-4690-887c-bfb592bab3a6 admin admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:32:53.366389 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686b35c3-007e-4690-887c-bfb592bab3a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:32:53.366737 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686b35c3-007e-4690-887c-bfb592bab3a6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:32:53.368139 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-686b35c3-007e-4690-887c-bfb592bab3a6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:32:53.369935 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-686b35c3-007e-4690-887c-bfb592bab3a6 admin admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:32:53.371480 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:32:53.384255 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1050/2093] 10.222.0.49 () {58 vars in 1121 bytes} [Wed Jun 15 10:32:53 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 10:32:53.385253 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686b35c3-007e-4690-887c-bfb592bab3a6 admin admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:32:53.385367 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39621dab-0ff0-4cf8-86f8-ace909fac62e service cinder] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:32:53.390670 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b99d14a-3402-489f-acd3-a4366d1191d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:32:53.391404 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b99d14a-3402-489f-acd3-a4366d1191d5 admin admin] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:32:53.391689 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b99d14a-3402-489f-acd3-a4366d1191d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:32:53.391968 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b99d14a-3402-489f-acd3-a4366d1191d5 admin admin] PATH_INFO: `/v3/users` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:32:53.392666 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b99d14a-3402-489f-acd3-a4366d1191d5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:32:53.394586 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b99d14a-3402-489f-acd3-a4366d1191d5 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:32:53.396232 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686b35c3-007e-4690-887c-bfb592bab3a6 admin admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:32:53.400926 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a8d4ec-6a0e-4660-9ce4-76e1bd01220f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:32:53.401972 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a8d4ec-6a0e-4660-9ce4-76e1bd01220f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:32:53.402320 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a8d4ec-6a0e-4660-9ce4-76e1bd01220f admin admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:32:53.402655 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a8d4ec-6a0e-4660-9ce4-76e1bd01220f admin admin] PATH_INFO: `/v3/users/25b88b19c2784a7b987fa4904c951c75` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:32:53.404344 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a8d4ec-6a0e-4660-9ce4-76e1bd01220f admin admin] RBAC: Authorizing `identity:delete_user(user_id=25b88b19c2784a7b987fa4904c951c75)` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:32:53.406868 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a8d4ec-6a0e-4660-9ce4-76e1bd01220f admin admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:32:53.415863 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1044/2094] 10.222.0.49 () {60 vars in 1143 bytes} [Wed Jun 15 10:32:53 2022] POST /identity/v3/users => generated 292 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 10:32:53.416702 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b99d14a-3402-489f-acd3-a4366d1191d5 admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:32:53.426631 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b99d14a-3402-489f-acd3-a4366d1191d5 admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:32:53.438890 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7dfed77-6758-4155-88b6-836de0bc7056 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:32:53.439967 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7dfed77-6758-4155-88b6-836de0bc7056 admin admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:32:53.440142 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7dfed77-6758-4155-88b6-836de0bc7056 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:32:53.440419 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7dfed77-6758-4155-88b6-836de0bc7056 admin admin] PATH_INFO: `/v3/roles` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:32:53.441842 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7dfed77-6758-4155-88b6-836de0bc7056 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:32:53.443695 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7dfed77-6758-4155-88b6-836de0bc7056 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:32:53.455125 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1045/2095] 10.222.0.49 () {58 vars in 1121 bytes} [Wed Jun 15 10:32:53 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 10:32:53.456066 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7dfed77-6758-4155-88b6-836de0bc7056 admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:32:53.466844 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-04a8d4ec-6a0e-4660-9ce4-76e1bd01220f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '25b88b19c2784a7b987fa4904c951c75'} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:32:53.467592 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7dfed77-6758-4155-88b6-836de0bc7056 admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:32:53.474395 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-04a8d4ec-6a0e-4660-9ce4-76e1bd01220f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '25b88b19c2784a7b987fa4904c951c75'} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:32:53.475074 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77753e1-2c04-4b7d-8fb0-109f22bf0b44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:32:53.475239 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-04a8d4ec-6a0e-4660-9ce4-76e1bd01220f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '25b88b19c2784a7b987fa4904c951c75'} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:32:53.475739 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-04a8d4ec-6a0e-4660-9ce4-76e1bd01220f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '25b88b19c2784a7b987fa4904c951c75'} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:32:53.475847 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77753e1-2c04-4b7d-8fb0-109f22bf0b44 admin admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:32:53.476041 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77753e1-2c04-4b7d-8fb0-109f22bf0b44 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:32:53.476319 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77753e1-2c04-4b7d-8fb0-109f22bf0b44 admin admin] PATH_INFO: `/v3/roles` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:32:53.477611 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a77753e1-2c04-4b7d-8fb0-109f22bf0b44 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:32:53.479303 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a77753e1-2c04-4b7d-8fb0-109f22bf0b44 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:32:53.484712 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-04a8d4ec-6a0e-4660-9ce4-76e1bd01220f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '25b88b19c2784a7b987fa4904c951c75'} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:32:53.492217 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1046/2096] 10.222.0.49 () {58 vars in 1121 bytes} [Wed Jun 15 10:32:53 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 10:32:53.493833 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77753e1-2c04-4b7d-8fb0-109f22bf0b44 admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:32:53.496900 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-a77753e1-2c04-4b7d-8fb0-109f22bf0b44 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:32:53.504675 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1051/2097] 10.222.0.49 () {58 vars in 1223 bytes} [Wed Jun 15 10:32:53 2022] DELETE /identity/v3/users/25b88b19c2784a7b987fa4904c951c75 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:32:53.505876 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a8d4ec-6a0e-4660-9ce4-76e1bd01220f admin admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:32:53.508693 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-04a8d4ec-6a0e-4660-9ce4-76e1bd01220f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:32:53.510591 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77753e1-2c04-4b7d-8fb0-109f22bf0b44 admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:32:53.581479 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6174b5-9bde-4c64-9b7b-659e6a29fd9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:32:53.582387 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6174b5-9bde-4c64-9b7b-659e6a29fd9d admin admin] REQUEST_METHOD: `PUT` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:32:53.582676 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6174b5-9bde-4c64-9b7b-659e6a29fd9d admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:32:53.582958 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6174b5-9bde-4c64-9b7b-659e6a29fd9d admin admin] PATH_INFO: `/v3/projects/a09e71951ab44f32ba9dd01952322542/users/b9af86d51e1642159a8fc73cfb81b1cf/roles/6876e57cc51c448097ab109d35b2a3da` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:32:53.587586 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a8d4ec-6a0e-4660-9ce4-76e1bd01220f admin admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:32:53.592827 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202384b9-3c29-4d0d-855b-2e84454f978b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:32:53.593713 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202384b9-3c29-4d0d-855b-2e84454f978b admin admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:32:53.594277 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202384b9-3c29-4d0d-855b-2e84454f978b admin admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:32:53.594639 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202384b9-3c29-4d0d-855b-2e84454f978b admin admin] PATH_INFO: `/v3/roles` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:32:53.596724 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-202384b9-3c29-4d0d-855b-2e84454f978b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:32:53.598660 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-202384b9-3c29-4d0d-855b-2e84454f978b admin admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:32:53.605577 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6174b5-9bde-4c64-9b7b-659e6a29fd9d admin admin] RBAC: Authorizing `identity:create_grant(project_id=a09e71951ab44f32ba9dd01952322542, user_id=b9af86d51e1642159a8fc73cfb81b1cf, role_id=6876e57cc51c448097ab109d35b2a3da)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:32:53.610196 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6174b5-9bde-4c64-9b7b-659e6a29fd9d admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:32:53.612089 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1052/2098] 10.222.0.49 () {58 vars in 1121 bytes} [Wed Jun 15 10:32:53 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 10:32:53.613450 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6e10d3-ca93-42f5-8719-75500978a8d9 None None] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:32:53.613802 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6e10d3-ca93-42f5-8719-75500978a8d9 None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:32:53.614091 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6e10d3-ca93-42f5-8719-75500978a8d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:32:53.643185 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1047/2099] 10.222.0.49 () {60 vars in 1482 bytes} [Wed Jun 15 10:32:53 2022] PUT /identity/v3/projects/a09e71951ab44f32ba9dd01952322542/users/b9af86d51e1642159a8fc73cfb81b1cf/roles/6876e57cc51c448097ab109d35b2a3da => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:32:53.643412 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.auth.core [None req-ac6e10d3-ca93-42f5-8719-75500978a8d9 None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:32:53.643496 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6174b5-9bde-4c64-9b7b-659e6a29fd9d admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:32:53.643570 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-ac6e10d3-ca93-42f5-8719-75500978a8d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:32:53.716186 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:32:53.728826 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1053/2100] 10.222.0.49 () {58 vars in 957 bytes} [Wed Jun 15 10:32:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:32:53.728826 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9772d0-a668-4759-99bd-6fa255e52135 None None] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:32:53.728826 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9772d0-a668-4759-99bd-6fa255e52135 None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:32:53.728826 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9772d0-a668-4759-99bd-6fa255e52135 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:32:53.742990 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:32:53.750672 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6174b5-9bde-4c64-9b7b-659e6a29fd9d admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:32:53.757069 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02f8e9b-922c-468b-a3ed-8c89e061471a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:32:53.758439 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02f8e9b-922c-468b-a3ed-8c89e061471a admin admin] REQUEST_METHOD: `PUT` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:32:53.759062 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02f8e9b-922c-468b-a3ed-8c89e061471a admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:32:53.759461 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02f8e9b-922c-468b-a3ed-8c89e061471a admin admin] PATH_INFO: `/v3/projects/13c11364ff594f9597521c3e6f9c03bd/users/d441c82018634d9796732339914a8559/roles/6876e57cc51c448097ab109d35b2a3da` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:32:53.776541 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.auth.core [None req-9a9772d0-a668-4759-99bd-6fa255e52135 None None] MFA Rules not processed for user `b9af86d51e1642159a8fc73cfb81b1cf`. Rule list: `[]` (Enabled: `True`). {{(pid=80119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:32:53.777871 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-9a9772d0-a668-4759-99bd-6fa255e52135 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:32:53.791237 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c02f8e9b-922c-468b-a3ed-8c89e061471a admin admin] RBAC: Authorizing `identity:create_grant(project_id=13c11364ff594f9597521c3e6f9c03bd, user_id=d441c82018634d9796732339914a8559, role_id=6876e57cc51c448097ab109d35b2a3da)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:32:53.794938 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c02f8e9b-922c-468b-a3ed-8c89e061471a admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:32:53.840248 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1048/2101] 10.222.0.49 () {60 vars in 1482 bytes} [Wed Jun 15 10:32:53 2022] PUT /identity/v3/projects/13c11364ff594f9597521c3e6f9c03bd/users/d441c82018634d9796732339914a8559/roles/6876e57cc51c448097ab109d35b2a3da => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:32:53.841881 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02f8e9b-922c-468b-a3ed-8c89e061471a admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:32:53.843704 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-c02f8e9b-922c-468b-a3ed-8c89e061471a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:32:53.888273 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1054/2102] 10.222.0.49 () {58 vars in 957 bytes} [Wed Jun 15 10:32:53 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:32:53.891955 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc798b59-e6e8-4100-8766-856c053bfa22 None None] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:32:53.891955 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc798b59-e6e8-4100-8766-856c053bfa22 None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:32:53.891955 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc798b59-e6e8-4100-8766-856c053bfa22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:32:53.931496 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.auth.core [None req-cc798b59-e6e8-4100-8766-856c053bfa22 None None] MFA Rules not processed for user `d441c82018634d9796732339914a8559`. Rule list: `[]` (Enabled: `True`). {{(pid=80119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:32:53.933247 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-cc798b59-e6e8-4100-8766-856c053bfa22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:32:53.955578 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02f8e9b-922c-468b-a3ed-8c89e061471a admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:32:53.965226 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c479d6f4-5c1b-42bb-a0f2-3b39c8371071 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0cb44a17d205431793e9a8791e477a32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '559b3e713f5844b6ae37151d56dd419f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:32:53.966460 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c479d6f4-5c1b-42bb-a0f2-3b39c8371071 service neutron] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:32:53.966823 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c479d6f4-5c1b-42bb-a0f2-3b39c8371071 service neutron] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:32:53.967481 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c479d6f4-5c1b-42bb-a0f2-3b39c8371071 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:32:54.021152 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1055/2103] 10.222.0.49 () {58 vars in 957 bytes} [Wed Jun 15 10:32:53 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:32:54.022395 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc798b59-e6e8-4100-8766-856c053bfa22 None None] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:32:54.047992 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c479d6f4-5c1b-42bb-a0f2-3b39c8371071 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:32:54.049426 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c479d6f4-5c1b-42bb-a0f2-3b39c8371071 service neutron] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:32:54.059572 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1049/2104] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Jun 15 10:32:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 219 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:32:54.060308 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c479d6f4-5c1b-42bb-a0f2-3b39c8371071 service neutron] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:32:54.069980 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c479d6f4-5c1b-42bb-a0f2-3b39c8371071 service neutron] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:32:54.075454 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b361672-1baa-4c61-b6d9-5b3cd4d5cf67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:32:54.076118 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b361672-1baa-4c61-b6d9-5b3cd4d5cf67 admin admin] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:32:54.076399 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b361672-1baa-4c61-b6d9-5b3cd4d5cf67 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:32:54.076675 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b361672-1baa-4c61-b6d9-5b3cd4d5cf67 admin admin] PATH_INFO: `/v3/projects` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:32:54.077363 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b361672-1baa-4c61-b6d9-5b3cd4d5cf67 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:32:54.079406 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b361672-1baa-4c61-b6d9-5b3cd4d5cf67 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:32:54.105408 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc798b59-e6e8-4100-8766-856c053bfa22 None None] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:32:54.110987 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f307a5ce-b04b-4543-b262-31a18e8e9f91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:32:54.111696 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f307a5ce-b04b-4543-b262-31a18e8e9f91 admin admin] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:32:54.112046 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f307a5ce-b04b-4543-b262-31a18e8e9f91 admin admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:32:54.112292 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f307a5ce-b04b-4543-b262-31a18e8e9f91 admin admin] PATH_INFO: `/v3/projects` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:32:54.112999 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f307a5ce-b04b-4543-b262-31a18e8e9f91 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:32:54.114871 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1050/2105] 10.222.0.49 () {60 vars in 1152 bytes} [Wed Jun 15 10:32:54 2022] POST /identity/v3/projects => generated 383 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 10:32:54.118121 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f307a5ce-b04b-4543-b262-31a18e8e9f91 admin admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:32:54.118665 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b361672-1baa-4c61-b6d9-5b3cd4d5cf67 admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:32:54.154488 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1056/2106] 10.222.0.49 () {60 vars in 1152 bytes} [Wed Jun 15 10:32:54 2022] POST /identity/v3/projects => generated 369 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 10:32:54.159508 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f307a5ce-b04b-4543-b262-31a18e8e9f91 admin admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:32:54.172268 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:32:54.190146 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:32:54.203886 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b361672-1baa-4c61-b6d9-5b3cd4d5cf67 admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:32:54.238984 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f307a5ce-b04b-4543-b262-31a18e8e9f91 admin admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:32:54.245104 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b360d0a-d10f-4c06-a5e5-94fd5f9e6012 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:32:54.245807 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b360d0a-d10f-4c06-a5e5-94fd5f9e6012 admin admin] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:32:54.246093 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b360d0a-d10f-4c06-a5e5-94fd5f9e6012 admin admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:32:54.246368 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b360d0a-d10f-4c06-a5e5-94fd5f9e6012 admin admin] PATH_INFO: `/v3/users` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:32:54.247057 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b360d0a-d10f-4c06-a5e5-94fd5f9e6012 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:32:54.249034 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b360d0a-d10f-4c06-a5e5-94fd5f9e6012 admin admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:32:54.273078 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1057/2107] 10.222.0.49 () {60 vars in 1143 bytes} [Wed Jun 15 10:32:54 2022] POST /identity/v3/users => generated 291 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 10:32:54.276574 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7061e85a-401a-4087-8f50-bc1810c160ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:32:54.276766 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b360d0a-d10f-4c06-a5e5-94fd5f9e6012 admin admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:32:54.277710 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7061e85a-401a-4087-8f50-bc1810c160ba admin admin] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:32:54.278002 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7061e85a-401a-4087-8f50-bc1810c160ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:32:54.278447 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7061e85a-401a-4087-8f50-bc1810c160ba admin admin] PATH_INFO: `/v3/users` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:32:54.279173 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7061e85a-401a-4087-8f50-bc1810c160ba admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:32:54.281378 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7061e85a-401a-4087-8f50-bc1810c160ba admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:32:54.289526 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b360d0a-d10f-4c06-a5e5-94fd5f9e6012 admin admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:32:54.296564 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1013d31-58ff-4307-abb8-d3dd5d100986 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:32:54.297351 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1013d31-58ff-4307-abb8-d3dd5d100986 admin admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:32:54.297666 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1013d31-58ff-4307-abb8-d3dd5d100986 admin admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:32:54.297967 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1013d31-58ff-4307-abb8-d3dd5d100986 admin admin] PATH_INFO: `/v3/roles` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:32:54.298731 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1013d31-58ff-4307-abb8-d3dd5d100986 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:32:54.300652 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1013d31-58ff-4307-abb8-d3dd5d100986 admin admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:32:54.317015 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1058/2108] 10.222.0.49 () {58 vars in 1121 bytes} [Wed Jun 15 10:32:54 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 10:32:54.318163 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1013d31-58ff-4307-abb8-d3dd5d100986 admin admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:32:54.318409 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1051/2109] 10.222.0.49 () {60 vars in 1143 bytes} [Wed Jun 15 10:32:54 2022] POST /identity/v3/users => generated 298 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 10:32:54.322978 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7061e85a-401a-4087-8f50-bc1810c160ba admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:32:54.333813 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7061e85a-401a-4087-8f50-bc1810c160ba admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:32:54.338319 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1013d31-58ff-4307-abb8-d3dd5d100986 admin admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:32:54.339245 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f53987-9493-4e2a-ab99-221096b03fc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:32:54.340317 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f53987-9493-4e2a-ab99-221096b03fc9 admin admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:32:54.340613 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f53987-9493-4e2a-ab99-221096b03fc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:32:54.340893 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f53987-9493-4e2a-ab99-221096b03fc9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:32:54.341633 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f53987-9493-4e2a-ab99-221096b03fc9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:32:54.343056 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09e7553-3db7-470b-8dc4-8951aae4390d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:32:54.343277 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f53987-9493-4e2a-ab99-221096b03fc9 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:32:54.343974 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09e7553-3db7-470b-8dc4-8951aae4390d admin admin] REQUEST_METHOD: `PUT` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:32:54.344262 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09e7553-3db7-470b-8dc4-8951aae4390d admin admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:32:54.344548 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09e7553-3db7-470b-8dc4-8951aae4390d admin admin] PATH_INFO: `/v3/projects/8904941a365c4760b656bf4ccc4bede8/users/586746a8e69d4994b25860202a25ebb5/roles/8051174e7a7947d5b991ebd793c2e543` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:32:54.354182 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1052/2110] 10.222.0.49 () {58 vars in 1121 bytes} [Wed Jun 15 10:32:54 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 15 10:32:54.358289 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f53987-9493-4e2a-ab99-221096b03fc9 admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:32:54.363154 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b09e7553-3db7-470b-8dc4-8951aae4390d admin admin] RBAC: Authorizing `identity:create_grant(project_id=8904941a365c4760b656bf4ccc4bede8, user_id=586746a8e69d4994b25860202a25ebb5, role_id=8051174e7a7947d5b991ebd793c2e543)` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:32:54.365924 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b09e7553-3db7-470b-8dc4-8951aae4390d admin admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:32:54.369024 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f53987-9493-4e2a-ab99-221096b03fc9 admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:32:54.374146 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787aa4c0-4765-40ea-9c56-248f73fe2cf0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:32:54.375077 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787aa4c0-4765-40ea-9c56-248f73fe2cf0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:32:54.375423 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787aa4c0-4765-40ea-9c56-248f73fe2cf0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:32:54.375751 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787aa4c0-4765-40ea-9c56-248f73fe2cf0 admin admin] PATH_INFO: `/v3/projects/28ef052d0a624e69ac7e8798ef76235e/users/1be0f3a7b0c74b0fbbd0ed6c401681bf/roles/8051174e7a7947d5b991ebd793c2e543` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:32:54.387389 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1059/2111] 10.222.0.49 () {60 vars in 1482 bytes} [Wed Jun 15 10:32:54 2022] PUT /identity/v3/projects/8904941a365c4760b656bf4ccc4bede8/users/586746a8e69d4994b25860202a25ebb5/roles/8051174e7a7947d5b991ebd793c2e543 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 10:32:54.388662 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09e7553-3db7-470b-8dc4-8951aae4390d admin admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:32:54.401972 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-787aa4c0-4765-40ea-9c56-248f73fe2cf0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=28ef052d0a624e69ac7e8798ef76235e, user_id=1be0f3a7b0c74b0fbbd0ed6c401681bf, role_id=8051174e7a7947d5b991ebd793c2e543)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:32:54.406181 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-787aa4c0-4765-40ea-9c56-248f73fe2cf0 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:32:54.421614 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1053/2112] 10.222.0.49 () {60 vars in 1482 bytes} [Wed Jun 15 10:32:54 2022] PUT /identity/v3/projects/28ef052d0a624e69ac7e8798ef76235e/users/1be0f3a7b0c74b0fbbd0ed6c401681bf/roles/8051174e7a7947d5b991ebd793c2e543 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 10:32:54.426965 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787aa4c0-4765-40ea-9c56-248f73fe2cf0 admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:32:54.468875 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09e7553-3db7-470b-8dc4-8951aae4390d admin admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:32:54.501956 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787aa4c0-4765-40ea-9c56-248f73fe2cf0 admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:32:54.507014 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0795f5dd-c985-487c-b197-faf1ec89aa27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:32:54.507920 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0795f5dd-c985-487c-b197-faf1ec89aa27 admin admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:32:54.508765 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0795f5dd-c985-487c-b197-faf1ec89aa27 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:32:54.509049 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0795f5dd-c985-487c-b197-faf1ec89aa27 admin admin] PATH_INFO: `/v3/roles` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:32:54.509788 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0795f5dd-c985-487c-b197-faf1ec89aa27 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:32:54.511430 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0795f5dd-c985-487c-b197-faf1ec89aa27 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:32:54.521983 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1054/2113] 10.222.0.49 () {58 vars in 1121 bytes} [Wed Jun 15 10:32:54 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 10:32:54.522614 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0795f5dd-c985-487c-b197-faf1ec89aa27 admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:32:54.524413 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-0795f5dd-c985-487c-b197-faf1ec89aa27 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:32:54.537397 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0795f5dd-c985-487c-b197-faf1ec89aa27 admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:32:54.542253 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9c7601-9a46-4e44-ac30-6b54001f9318 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:32:54.543114 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9c7601-9a46-4e44-ac30-6b54001f9318 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:32:54.543402 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9c7601-9a46-4e44-ac30-6b54001f9318 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:32:54.543679 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9c7601-9a46-4e44-ac30-6b54001f9318 admin admin] PATH_INFO: `/v3/projects/f8036511910d43eab14c26921b0d730c` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:32:54.544874 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff9c7601-9a46-4e44-ac30-6b54001f9318 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f8036511910d43eab14c26921b0d730c)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:32:54.545841 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd4d80a-49c2-41e8-bdaf-30dddef2178e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:32:54.546517 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd4d80a-49c2-41e8-bdaf-30dddef2178e admin admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:32:54.546772 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff9c7601-9a46-4e44-ac30-6b54001f9318 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:32:54.546880 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd4d80a-49c2-41e8-bdaf-30dddef2178e admin admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:32:54.547075 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd4d80a-49c2-41e8-bdaf-30dddef2178e admin admin] PATH_INFO: `/v3/roles` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:32:54.547837 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd4d80a-49c2-41e8-bdaf-30dddef2178e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:32:54.549463 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd4d80a-49c2-41e8-bdaf-30dddef2178e admin admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:32:54.560873 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1060/2114] 10.222.0.49 () {58 vars in 1121 bytes} [Wed Jun 15 10:32:54 2022] GET /identity/v3/roles => generated 1420 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 10:32:54.561047 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd4d80a-49c2-41e8-bdaf-30dddef2178e admin admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:32:54.577013 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd4d80a-49c2-41e8-bdaf-30dddef2178e admin admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:32:54.582876 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736eac1d-a23b-4efb-ad1b-d3b78e2ccdff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:32:54.583888 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736eac1d-a23b-4efb-ad1b-d3b78e2ccdff admin admin] REQUEST_METHOD: `PUT` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:32:54.584284 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736eac1d-a23b-4efb-ad1b-d3b78e2ccdff admin admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:32:54.584671 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736eac1d-a23b-4efb-ad1b-d3b78e2ccdff admin admin] PATH_INFO: `/v3/projects/28ef052d0a624e69ac7e8798ef76235e/users/1be0f3a7b0c74b0fbbd0ed6c401681bf/roles/8051174e7a7947d5b991ebd793c2e543` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:32:54.587066 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-736eac1d-a23b-4efb-ad1b-d3b78e2ccdff admin admin] RBAC: Authorizing `identity:create_grant(project_id=28ef052d0a624e69ac7e8798ef76235e, user_id=1be0f3a7b0c74b0fbbd0ed6c401681bf, role_id=8051174e7a7947d5b991ebd793c2e543)` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:32:54.590954 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-736eac1d-a23b-4efb-ad1b-d3b78e2ccdff admin admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:32:54.593524 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:32:54.602829 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-ff9c7601-9a46-4e44-ac30-6b54001f9318 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f8036511910d43eab14c26921b0d730c', 'request_id': 'req-ff9c7601-9a46-4e44-ac30-6b54001f9318'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:32:54.604695 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=92,keystone:INSERT=4,keystone:DELETE=4 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:32:54.605043 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1061/2115] 10.222.0.49 () {60 vars in 1482 bytes} [Wed Jun 15 10:32:54 2022] PUT /identity/v3/projects/28ef052d0a624e69ac7e8798ef76235e/users/1be0f3a7b0c74b0fbbd0ed6c401681bf/roles/8051174e7a7947d5b991ebd793c2e543 => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:32:54.605910 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736eac1d-a23b-4efb-ad1b-d3b78e2ccdff admin admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:32:54.619362 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-ff9c7601-9a46-4e44-ac30-6b54001f9318 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f8036511910d43eab14c26921b0d730c', 'request_id': 'req-ff9c7601-9a46-4e44-ac30-6b54001f9318'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:32:54.626803 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-ff9c7601-9a46-4e44-ac30-6b54001f9318 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f8036511910d43eab14c26921b0d730c', 'request_id': 'req-ff9c7601-9a46-4e44-ac30-6b54001f9318'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:32:54.639100 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-ff9c7601-9a46-4e44-ac30-6b54001f9318 admin admin] The token cache is being invalidate because project f8036511910d43eab14c26921b0d730c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80120) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 10:32:54.639568 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-ff9c7601-9a46-4e44-ac30-6b54001f9318 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:32:54.640132 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-ff9c7601-9a46-4e44-ac30-6b54001f9318 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:32:54.641700 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1055/2116] 10.222.0.49 () {58 vars in 1232 bytes} [Wed Jun 15 10:32:54 2022] DELETE /identity/v3/projects/f8036511910d43eab14c26921b0d730c => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:32:54.642941 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651b3821-8639-4089-afe1-d96cfe6f0f8a None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:32:54.643364 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651b3821-8639-4089-afe1-d96cfe6f0f8a None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:32:54.643728 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651b3821-8639-4089-afe1-d96cfe6f0f8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:32:54.679222 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-651b3821-8639-4089-afe1-d96cfe6f0f8a None None] MFA Rules not processed for user `1be0f3a7b0c74b0fbbd0ed6c401681bf`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:32:54.682297 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-651b3821-8639-4089-afe1-d96cfe6f0f8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:32:54.701399 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736eac1d-a23b-4efb-ad1b-d3b78e2ccdff admin admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:32:54.706976 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9b3806-3fc9-4067-8995-1cd88def4e38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:32:54.708214 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9b3806-3fc9-4067-8995-1cd88def4e38 admin admin] REQUEST_METHOD: `PUT` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:32:54.708506 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9b3806-3fc9-4067-8995-1cd88def4e38 admin admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:32:54.708798 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9b3806-3fc9-4067-8995-1cd88def4e38 admin admin] PATH_INFO: `/v3/projects/8904941a365c4760b656bf4ccc4bede8/users/586746a8e69d4994b25860202a25ebb5/roles/8051174e7a7947d5b991ebd793c2e543` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:32:54.713823 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b9b3806-3fc9-4067-8995-1cd88def4e38 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8904941a365c4760b656bf4ccc4bede8, user_id=586746a8e69d4994b25860202a25ebb5, role_id=8051174e7a7947d5b991ebd793c2e543)` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:32:54.714253 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b9b3806-3fc9-4067-8995-1cd88def4e38 admin admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:32:54.739065 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1062/2117] 10.222.0.49 () {60 vars in 1482 bytes} [Wed Jun 15 10:32:54 2022] PUT /identity/v3/projects/8904941a365c4760b656bf4ccc4bede8/users/586746a8e69d4994b25860202a25ebb5/roles/8051174e7a7947d5b991ebd793c2e543 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:32:54.750994 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aace9b8-231c-4589-b8ac-0607772dccc9 None None] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:32:54.752672 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aace9b8-231c-4589-b8ac-0607772dccc9 None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:32:54.752672 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aace9b8-231c-4589-b8ac-0607772dccc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:32:54.791884 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.auth.core [None req-1aace9b8-231c-4589-b8ac-0607772dccc9 None None] MFA Rules not processed for user `586746a8e69d4994b25860202a25ebb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:32:54.793582 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-1aace9b8-231c-4589-b8ac-0607772dccc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:32:54.807134 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1056/2118] 10.222.0.49 () {58 vars in 957 bytes} [Wed Jun 15 10:32:54 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:32:54.819133 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651b3821-8639-4089-afe1-d96cfe6f0f8a None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:32:54.820818 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-651b3821-8639-4089-afe1-d96cfe6f0f8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:32:54.897963 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651b3821-8639-4089-afe1-d96cfe6f0f8a None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:32:54.903983 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1063/2119] 10.222.0.49 () {58 vars in 957 bytes} [Wed Jun 15 10:32:54 2022] POST /identity/v3/auth/tokens => generated 3303 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:32:54.915468 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aace9b8-231c-4589-b8ac-0607772dccc9 None None] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:32:54.916837 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-1aace9b8-231c-4589-b8ac-0607772dccc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:32:54.963394 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e21906-b3bc-4ce1-b460-77abdbe8f489 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20d1327553404b80aecee87c78fcb52b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '559b3e713f5844b6ae37151d56dd419f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:32:54.964563 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e21906-b3bc-4ce1-b460-77abdbe8f489 service cinder] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:32:54.965060 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e21906-b3bc-4ce1-b460-77abdbe8f489 service cinder] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:32:54.965519 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e21906-b3bc-4ce1-b460-77abdbe8f489 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:32:54.968000 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-a1e21906-b3bc-4ce1-b460-77abdbe8f489 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:32:54.991546 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aace9b8-231c-4589-b8ac-0607772dccc9 None None] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:32:54.996493 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7845aca8-8a35-4972-9a02-0c45eae04eb6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20d1327553404b80aecee87c78fcb52b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '559b3e713f5844b6ae37151d56dd419f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:32:54.997375 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7845aca8-8a35-4972-9a02-0c45eae04eb6 service cinder] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:32:54.997663 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7845aca8-8a35-4972-9a02-0c45eae04eb6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:32:54.997937 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7845aca8-8a35-4972-9a02-0c45eae04eb6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:32:55.013740 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7845aca8-8a35-4972-9a02-0c45eae04eb6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:32:55.015306 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7845aca8-8a35-4972-9a02-0c45eae04eb6 service cinder] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:32:55.018511 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:INSERT=1,keystone:DELETE=3,keystone:SELECT=96 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:32:55.026693 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1064/2120] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Jun 15 10:32:54 2022] GET /identity/v3/auth/tokens => generated 3303 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 10:32:55.050527 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e21906-b3bc-4ce1-b460-77abdbe8f489 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:32:55.051993 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e21906-b3bc-4ce1-b460-77abdbe8f489 service cinder] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:32:55.061417 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1057/2121] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Jun 15 10:32:54 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 243 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:32:56.521665 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7845aca8-8a35-4972-9a02-0c45eae04eb6 service cinder] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:32:56.523334 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-7845aca8-8a35-4972-9a02-0c45eae04eb6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:32:56.544057 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:32:56.593723 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e21906-b3bc-4ce1-b460-77abdbe8f489 service cinder] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:32:56.596570 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7845aca8-8a35-4972-9a02-0c45eae04eb6 service cinder] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:32:56.601832 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2327f0fb-a445-4ffb-9ae4-fb0241b79e88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:32:56.602709 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2327f0fb-a445-4ffb-9ae4-fb0241b79e88 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:32:56.602993 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e21906-b3bc-4ce1-b460-77abdbe8f489 service cinder] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:32:56.603222 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2327f0fb-a445-4ffb-9ae4-fb0241b79e88 admin admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:32:56.603439 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2327f0fb-a445-4ffb-9ae4-fb0241b79e88 admin admin] PATH_INFO: `/v3/users/b9af86d51e1642159a8fc73cfb81b1cf` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:32:56.604528 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2327f0fb-a445-4ffb-9ae4-fb0241b79e88 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b9af86d51e1642159a8fc73cfb81b1cf)` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:32:56.609151 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2327f0fb-a445-4ffb-9ae4-fb0241b79e88 admin admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:32:56.611209 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b48a82-9af6-40c0-b33d-ba6da2bce912 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20d1327553404b80aecee87c78fcb52b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '559b3e713f5844b6ae37151d56dd419f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:32:56.611890 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b48a82-9af6-40c0-b33d-ba6da2bce912 service cinder] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:32:56.612176 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b48a82-9af6-40c0-b33d-ba6da2bce912 service cinder] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:32:56.612451 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b48a82-9af6-40c0-b33d-ba6da2bce912 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:32:56.615337 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-d0b48a82-9af6-40c0-b33d-ba6da2bce912 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:32:56.659024 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-2327f0fb-a445-4ffb-9ae4-fb0241b79e88 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b9af86d51e1642159a8fc73cfb81b1cf'} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:32:56.665977 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-2327f0fb-a445-4ffb-9ae4-fb0241b79e88 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b9af86d51e1642159a8fc73cfb81b1cf'} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:32:56.666659 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-2327f0fb-a445-4ffb-9ae4-fb0241b79e88 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b9af86d51e1642159a8fc73cfb81b1cf'} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:32:56.667312 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-2327f0fb-a445-4ffb-9ae4-fb0241b79e88 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b9af86d51e1642159a8fc73cfb81b1cf'} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:32:56.676507 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b48a82-9af6-40c0-b33d-ba6da2bce912 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:32:56.677976 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b48a82-9af6-40c0-b33d-ba6da2bce912 service cinder] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:32:56.680416 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-2327f0fb-a445-4ffb-9ae4-fb0241b79e88 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b9af86d51e1642159a8fc73cfb81b1cf'} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:32:56.688467 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1058/2122] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Jun 15 10:32:56 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:32:56.699517 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1065/2123] 10.222.0.49 () {58 vars in 1223 bytes} [Wed Jun 15 10:32:56 2022] DELETE /identity/v3/users/b9af86d51e1642159a8fc73cfb81b1cf => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:32:56.703923 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5827c9b6-bc3c-466c-9431-9569292a6dd4 None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:32:56.704219 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5827c9b6-bc3c-466c-9431-9569292a6dd4 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:32:56.704499 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5827c9b6-bc3c-466c-9431-9569292a6dd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:32:56.721192 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-5827c9b6-bc3c-466c-9431-9569292a6dd4 None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:32:56.722453 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-5827c9b6-bc3c-466c-9431-9569292a6dd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:32:56.786455 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1059/2124] 10.222.0.49 () {58 vars in 957 bytes} [Wed Jun 15 10:32:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:32:56.797733 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2327f0fb-a445-4ffb-9ae4-fb0241b79e88 admin admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:32:56.799104 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-2327f0fb-a445-4ffb-9ae4-fb0241b79e88 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:32:56.865470 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2327f0fb-a445-4ffb-9ae4-fb0241b79e88 admin admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:32:56.870266 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cccccde8-4b84-458b-9e0c-81a150a8cd9f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0cb44a17d205431793e9a8791e477a32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '559b3e713f5844b6ae37151d56dd419f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:32:56.870924 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccccde8-4b84-458b-9e0c-81a150a8cd9f service neutron] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:32:56.871240 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccccde8-4b84-458b-9e0c-81a150a8cd9f service neutron] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:32:56.871518 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccccde8-4b84-458b-9e0c-81a150a8cd9f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:32:56.880773 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cccccde8-4b84-458b-9e0c-81a150a8cd9f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:32:56.882260 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cccccde8-4b84-458b-9e0c-81a150a8cd9f service neutron] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:32:56.892986 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1066/2125] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Jun 15 10:32:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 10:32:57.191920 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5827c9b6-bc3c-466c-9431-9569292a6dd4 None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:32:57.194233 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-5827c9b6-bc3c-466c-9431-9569292a6dd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:32:57.212394 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5827c9b6-bc3c-466c-9431-9569292a6dd4 None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:32:57.217147 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28b4be5-1cb9-455d-95fc-ffc7686a8051 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:32:57.218165 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28b4be5-1cb9-455d-95fc-ffc7686a8051 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:32:57.218455 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28b4be5-1cb9-455d-95fc-ffc7686a8051 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:32:57.218734 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28b4be5-1cb9-455d-95fc-ffc7686a8051 admin admin] PATH_INFO: `/v3/projects/a09e71951ab44f32ba9dd01952322542` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:32:57.220779 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b28b4be5-1cb9-455d-95fc-ffc7686a8051 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a09e71951ab44f32ba9dd01952322542)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:32:57.222134 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b28b4be5-1cb9-455d-95fc-ffc7686a8051 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:32:57.288785 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cccccde8-4b84-458b-9e0c-81a150a8cd9f service neutron] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:32:57.291078 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-cccccde8-4b84-458b-9e0c-81a150a8cd9f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:32:57.312707 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-b28b4be5-1cb9-455d-95fc-ffc7686a8051 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a09e71951ab44f32ba9dd01952322542', 'request_id': 'req-b28b4be5-1cb9-455d-95fc-ffc7686a8051'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:32:57.323973 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-b28b4be5-1cb9-455d-95fc-ffc7686a8051 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a09e71951ab44f32ba9dd01952322542', 'request_id': 'req-b28b4be5-1cb9-455d-95fc-ffc7686a8051'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:32:57.340456 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-b28b4be5-1cb9-455d-95fc-ffc7686a8051 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a09e71951ab44f32ba9dd01952322542', 'request_id': 'req-b28b4be5-1cb9-455d-95fc-ffc7686a8051'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:32:57.343083 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:32:57.347854 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:32:57.363047 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-b28b4be5-1cb9-455d-95fc-ffc7686a8051 admin admin] The token cache is being invalidate because project a09e71951ab44f32ba9dd01952322542 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80120) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 10:32:57.363375 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-b28b4be5-1cb9-455d-95fc-ffc7686a8051 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:32:57.364098 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-b28b4be5-1cb9-455d-95fc-ffc7686a8051 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:32:57.370483 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1060/2126] 10.222.0.49 () {58 vars in 1232 bytes} [Wed Jun 15 10:32:57 2022] DELETE /identity/v3/projects/a09e71951ab44f32ba9dd01952322542 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:32:57.371173 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cccccde8-4b84-458b-9e0c-81a150a8cd9f service neutron] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:32:57.371516 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28b4be5-1cb9-455d-95fc-ffc7686a8051 admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:32:57.373948 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-b28b4be5-1cb9-455d-95fc-ffc7686a8051 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:32:57.448579 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848e4cd3-a0d1-4795-ac80-764c0a46b4ba service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20d1327553404b80aecee87c78fcb52b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '559b3e713f5844b6ae37151d56dd419f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:32:57.449257 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848e4cd3-a0d1-4795-ac80-764c0a46b4ba service cinder] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:32:57.449542 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848e4cd3-a0d1-4795-ac80-764c0a46b4ba service cinder] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:32:57.449992 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848e4cd3-a0d1-4795-ac80-764c0a46b4ba service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:32:57.452199 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-848e4cd3-a0d1-4795-ac80-764c0a46b4ba service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:32:57.487638 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28b4be5-1cb9-455d-95fc-ffc7686a8051 admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:32:57.499668 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad82933-e978-4edc-8836-4c046ffc57af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:32:57.500514 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad82933-e978-4edc-8836-4c046ffc57af admin admin] REQUEST_METHOD: `DELETE` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:32:57.500798 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad82933-e978-4edc-8836-4c046ffc57af admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:32:57.501076 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad82933-e978-4edc-8836-4c046ffc57af admin admin] PATH_INFO: `/v3/users/586746a8e69d4994b25860202a25ebb5` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:32:57.502319 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad82933-e978-4edc-8836-4c046ffc57af admin admin] RBAC: Authorizing `identity:delete_user(user_id=586746a8e69d4994b25860202a25ebb5)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:32:57.504439 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad82933-e978-4edc-8836-4c046ffc57af admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:32:57.539860 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848e4cd3-a0d1-4795-ac80-764c0a46b4ba service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:32:57.541342 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848e4cd3-a0d1-4795-ac80-764c0a46b4ba service cinder] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:32:57.552476 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1067/2127] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Jun 15 10:32:57 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 264 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:32:57.592716 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-3ad82933-e978-4edc-8836-4c046ffc57af admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '586746a8e69d4994b25860202a25ebb5'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:32:57.602553 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-3ad82933-e978-4edc-8836-4c046ffc57af admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '586746a8e69d4994b25860202a25ebb5'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:32:57.603772 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-3ad82933-e978-4edc-8836-4c046ffc57af admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '586746a8e69d4994b25860202a25ebb5'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:32:57.604615 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-3ad82933-e978-4edc-8836-4c046ffc57af admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '586746a8e69d4994b25860202a25ebb5'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:32:57.616514 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-3ad82933-e978-4edc-8836-4c046ffc57af admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '586746a8e69d4994b25860202a25ebb5'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:32:57.639657 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1061/2128] 10.222.0.49 () {58 vars in 1223 bytes} [Wed Jun 15 10:32:57 2022] DELETE /identity/v3/users/586746a8e69d4994b25860202a25ebb5 => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:32:58.188232 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848e4cd3-a0d1-4795-ac80-764c0a46b4ba service cinder] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:32:58.199825 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848e4cd3-a0d1-4795-ac80-764c0a46b4ba service cinder] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:32:58.287834 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1944e828-d6be-40c8-be34-ee6647b349fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:32:58.288826 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1944e828-d6be-40c8-be34-ee6647b349fb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:32:58.289248 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1944e828-d6be-40c8-be34-ee6647b349fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:32:58.290420 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1944e828-d6be-40c8-be34-ee6647b349fb admin admin] PATH_INFO: `/v3/projects/8904941a365c4760b656bf4ccc4bede8` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:32:58.291502 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1944e828-d6be-40c8-be34-ee6647b349fb admin admin] RBAC: Authorizing `identity:delete_project(project_id=8904941a365c4760b656bf4ccc4bede8)` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:32:58.293754 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1944e828-d6be-40c8-be34-ee6647b349fb admin admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:32:58.347075 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:32:58.350258 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-1944e828-d6be-40c8-be34-ee6647b349fb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8904941a365c4760b656bf4ccc4bede8', 'request_id': 'req-1944e828-d6be-40c8-be34-ee6647b349fb'} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:32:58.358895 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-1944e828-d6be-40c8-be34-ee6647b349fb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8904941a365c4760b656bf4ccc4bede8', 'request_id': 'req-1944e828-d6be-40c8-be34-ee6647b349fb'} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:32:58.364804 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-1944e828-d6be-40c8-be34-ee6647b349fb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8904941a365c4760b656bf4ccc4bede8', 'request_id': 'req-1944e828-d6be-40c8-be34-ee6647b349fb'} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:32:58.376037 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-1944e828-d6be-40c8-be34-ee6647b349fb admin admin] The token cache is being invalidate because project 8904941a365c4760b656bf4ccc4bede8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80119) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 10:32:58.376341 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-1944e828-d6be-40c8-be34-ee6647b349fb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:32:58.377203 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-1944e828-d6be-40c8-be34-ee6647b349fb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:32:58.378996 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1068/2129] 10.222.0.49 () {58 vars in 1232 bytes} [Wed Jun 15 10:32:58 2022] DELETE /identity/v3/projects/8904941a365c4760b656bf4ccc4bede8 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:32:58.384561 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c5f12d-a639-4a4e-987f-28c92dd72bd3 None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:32:58.384856 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c5f12d-a639-4a4e-987f-28c92dd72bd3 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:32:58.385139 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c5f12d-a639-4a4e-987f-28c92dd72bd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:32:58.406617 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-16c5f12d-a639-4a4e-987f-28c92dd72bd3 None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:32:58.407881 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-16c5f12d-a639-4a4e-987f-28c92dd72bd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:32:58.467669 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1062/2130] 10.222.0.49 () {58 vars in 957 bytes} [Wed Jun 15 10:32:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:32:58.475424 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016c5bc8-4241-4281-bc47-ec3d3b5987ca None None] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:32:58.475719 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016c5bc8-4241-4281-bc47-ec3d3b5987ca None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:32:58.476005 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016c5bc8-4241-4281-bc47-ec3d3b5987ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:32:58.494393 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.auth.core [None req-016c5bc8-4241-4281-bc47-ec3d3b5987ca None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:32:58.495653 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-016c5bc8-4241-4281-bc47-ec3d3b5987ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:32:58.504144 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1069/2131] 10.222.0.49 () {58 vars in 957 bytes} [Wed Jun 15 10:32:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:32:58.508950 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c5f12d-a639-4a4e-987f-28c92dd72bd3 None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:32:58.519034 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c5f12d-a639-4a4e-987f-28c92dd72bd3 None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:32:58.524489 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92edaf3-7879-44e5-b2ed-036e7fe8f768 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:32:58.525199 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92edaf3-7879-44e5-b2ed-036e7fe8f768 admin admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:32:58.525505 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92edaf3-7879-44e5-b2ed-036e7fe8f768 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:32:58.525786 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92edaf3-7879-44e5-b2ed-036e7fe8f768 admin admin] PATH_INFO: `/v3/domains` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:32:58.526532 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b92edaf3-7879-44e5-b2ed-036e7fe8f768 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:32:58.526989 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b92edaf3-7879-44e5-b2ed-036e7fe8f768 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:32:58.528681 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b92edaf3-7879-44e5-b2ed-036e7fe8f768 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:32:58.546380 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1063/2132] 10.222.0.49 () {58 vars in 1152 bytes} [Wed Jun 15 10:32:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:32:58.550886 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016c5bc8-4241-4281-bc47-ec3d3b5987ca None None] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:32:58.560715 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016c5bc8-4241-4281-bc47-ec3d3b5987ca None None] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:32:58.565478 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3689c9b5-2307-4b52-9cd9-67be752d5d6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:32:58.566157 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3689c9b5-2307-4b52-9cd9-67be752d5d6c admin admin] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:32:58.567221 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3689c9b5-2307-4b52-9cd9-67be752d5d6c admin admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:32:58.567511 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3689c9b5-2307-4b52-9cd9-67be752d5d6c admin admin] PATH_INFO: `/v3/projects` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:32:58.568215 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3689c9b5-2307-4b52-9cd9-67be752d5d6c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:32:58.570695 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3689c9b5-2307-4b52-9cd9-67be752d5d6c admin admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:32:58.604798 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1070/2133] 10.222.0.49 () {60 vars in 1152 bytes} [Wed Jun 15 10:32:58 2022] POST /identity/v3/projects => generated 385 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 10:32:58.609312 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92edaf3-7879-44e5-b2ed-036e7fe8f768 admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:32:58.614781 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:32:58.620565 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92edaf3-7879-44e5-b2ed-036e7fe8f768 admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:32:58.687576 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7e2f88-38e7-4d7a-8079-19a399aac459 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:32:58.688266 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7e2f88-38e7-4d7a-8079-19a399aac459 admin admin] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:32:58.688266 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7e2f88-38e7-4d7a-8079-19a399aac459 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:32:58.688559 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7e2f88-38e7-4d7a-8079-19a399aac459 admin admin] PATH_INFO: `/v3/users` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:32:58.689210 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7e2f88-38e7-4d7a-8079-19a399aac459 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:32:58.691438 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7e2f88-38e7-4d7a-8079-19a399aac459 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:32:58.713948 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1064/2134] 10.222.0.49 () {60 vars in 1143 bytes} [Wed Jun 15 10:32:58 2022] POST /identity/v3/users => generated 293 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 10:32:58.718418 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3689c9b5-2307-4b52-9cd9-67be752d5d6c admin admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:32:58.730322 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3689c9b5-2307-4b52-9cd9-67be752d5d6c admin admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:32:58.736067 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ada1858-8128-4eb6-97c1-cff9db330d39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:32:58.736746 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ada1858-8128-4eb6-97c1-cff9db330d39 admin admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:32:58.737037 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ada1858-8128-4eb6-97c1-cff9db330d39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:32:58.737328 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ada1858-8128-4eb6-97c1-cff9db330d39 admin admin] PATH_INFO: `/v3/roles` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:32:58.738051 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ada1858-8128-4eb6-97c1-cff9db330d39 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:32:58.739718 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ada1858-8128-4eb6-97c1-cff9db330d39 admin admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:32:58.756993 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1071/2135] 10.222.0.49 () {58 vars in 1121 bytes} [Wed Jun 15 10:32:58 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 15 10:32:58.761240 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7e2f88-38e7-4d7a-8079-19a399aac459 admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:32:58.783444 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7e2f88-38e7-4d7a-8079-19a399aac459 admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:32:58.797407 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19bbe679-9c81-41bc-b13a-fdf249455149 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:32:58.798310 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bbe679-9c81-41bc-b13a-fdf249455149 admin admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:32:58.798570 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bbe679-9c81-41bc-b13a-fdf249455149 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:32:58.798885 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bbe679-9c81-41bc-b13a-fdf249455149 admin admin] PATH_INFO: `/v3/roles` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:32:58.799836 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19bbe679-9c81-41bc-b13a-fdf249455149 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:32:58.801583 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19bbe679-9c81-41bc-b13a-fdf249455149 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:32:58.816489 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1065/2136] 10.222.0.49 () {58 vars in 1121 bytes} [Wed Jun 15 10:32:58 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 10:32:58.818035 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ada1858-8128-4eb6-97c1-cff9db330d39 admin admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:32:58.829663 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ada1858-8128-4eb6-97c1-cff9db330d39 admin admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:32:58.835200 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e34da94-df1c-46b0-85ea-fe3a1a5f794e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:32:58.836809 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e34da94-df1c-46b0-85ea-fe3a1a5f794e admin admin] REQUEST_METHOD: `PUT` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:32:58.837192 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e34da94-df1c-46b0-85ea-fe3a1a5f794e admin admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:32:58.837674 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e34da94-df1c-46b0-85ea-fe3a1a5f794e admin admin] PATH_INFO: `/v3/projects/a36746ae425246beafadf564a07d8648/users/5aba4ac9eed84879b00c0eea0cc39c72/roles/6876e57cc51c448097ab109d35b2a3da` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:32:58.860093 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e34da94-df1c-46b0-85ea-fe3a1a5f794e admin admin] RBAC: Authorizing `identity:create_grant(project_id=a36746ae425246beafadf564a07d8648, user_id=5aba4ac9eed84879b00c0eea0cc39c72, role_id=6876e57cc51c448097ab109d35b2a3da)` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:32:58.863765 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e34da94-df1c-46b0-85ea-fe3a1a5f794e admin admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:32:58.889663 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1072/2137] 10.222.0.49 () {60 vars in 1482 bytes} [Wed Jun 15 10:32:58 2022] PUT /identity/v3/projects/a36746ae425246beafadf564a07d8648/users/5aba4ac9eed84879b00c0eea0cc39c72/roles/6876e57cc51c448097ab109d35b2a3da => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 10:32:58.900844 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d50142-b9ef-46a3-aae5-85da51084dc8 None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:32:58.901185 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d50142-b9ef-46a3-aae5-85da51084dc8 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:32:58.902002 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d50142-b9ef-46a3-aae5-85da51084dc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:32:58.949122 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-35d50142-b9ef-46a3-aae5-85da51084dc8 None None] MFA Rules not processed for user `5aba4ac9eed84879b00c0eea0cc39c72`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:32:58.950838 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-35d50142-b9ef-46a3-aae5-85da51084dc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:32:59.054137 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1066/2138] 10.222.0.49 () {58 vars in 957 bytes} [Wed Jun 15 10:32:58 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:32:59.054337 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e34da94-df1c-46b0-85ea-fe3a1a5f794e admin admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:32:59.150048 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e34da94-df1c-46b0-85ea-fe3a1a5f794e admin admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:32:59.156209 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149387b0-e535-41c6-8e77-4af659449b8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:32:59.158858 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149387b0-e535-41c6-8e77-4af659449b8b admin admin] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:32:59.160293 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149387b0-e535-41c6-8e77-4af659449b8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:32:59.160943 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149387b0-e535-41c6-8e77-4af659449b8b admin admin] PATH_INFO: `/v3/projects` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:32:59.161764 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-149387b0-e535-41c6-8e77-4af659449b8b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:32:59.164533 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-149387b0-e535-41c6-8e77-4af659449b8b admin admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:32:59.193158 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1073/2139] 10.222.0.49 () {60 vars in 1152 bytes} [Wed Jun 15 10:32:59 2022] POST /identity/v3/projects => generated 387 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 10:32:59.194423 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d50142-b9ef-46a3-aae5-85da51084dc8 None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:32:59.207012 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d50142-b9ef-46a3-aae5-85da51084dc8 None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:32:59.266876 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5095be68-2ab5-44e9-96d3-0531768ca09a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:32:59.267485 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5095be68-2ab5-44e9-96d3-0531768ca09a admin admin] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:32:59.267772 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5095be68-2ab5-44e9-96d3-0531768ca09a admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:32:59.268043 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5095be68-2ab5-44e9-96d3-0531768ca09a admin admin] PATH_INFO: `/v3/users` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:32:59.268733 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5095be68-2ab5-44e9-96d3-0531768ca09a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:32:59.270651 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5095be68-2ab5-44e9-96d3-0531768ca09a admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:32:59.286902 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:32:59.295345 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1067/2140] 10.222.0.49 () {60 vars in 1143 bytes} [Wed Jun 15 10:32:59 2022] POST /identity/v3/users => generated 300 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 10:32:59.296733 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149387b0-e535-41c6-8e77-4af659449b8b admin admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:32:59.306604 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149387b0-e535-41c6-8e77-4af659449b8b admin admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:32:59.311266 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ddd2b41-d0d6-410f-80cc-1b16a6e5aa4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:32:59.311940 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddd2b41-d0d6-410f-80cc-1b16a6e5aa4b admin admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:32:59.312220 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddd2b41-d0d6-410f-80cc-1b16a6e5aa4b admin admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:32:59.312494 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddd2b41-d0d6-410f-80cc-1b16a6e5aa4b admin admin] PATH_INFO: `/v3/roles` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:32:59.313221 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ddd2b41-d0d6-410f-80cc-1b16a6e5aa4b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:32:59.314825 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ddd2b41-d0d6-410f-80cc-1b16a6e5aa4b admin admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:32:59.326152 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1074/2141] 10.222.0.49 () {58 vars in 1121 bytes} [Wed Jun 15 10:32:59 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 15 10:32:59.330064 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5095be68-2ab5-44e9-96d3-0531768ca09a admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:32:59.341618 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5095be68-2ab5-44e9-96d3-0531768ca09a admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:32:59.347007 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26aff30-9d6c-46c1-8344-4e2ecf44b326 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:32:59.347977 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26aff30-9d6c-46c1-8344-4e2ecf44b326 admin admin] REQUEST_METHOD: `PUT` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:32:59.348295 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26aff30-9d6c-46c1-8344-4e2ecf44b326 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:32:59.349954 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26aff30-9d6c-46c1-8344-4e2ecf44b326 admin admin] PATH_INFO: `/v3/projects/5c52d48547364b2f9bad7234a6189dab/users/6d656a8163e24d538a1e66af58b51998/roles/8051174e7a7947d5b991ebd793c2e543` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:32:59.366358 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b26aff30-9d6c-46c1-8344-4e2ecf44b326 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5c52d48547364b2f9bad7234a6189dab, user_id=6d656a8163e24d538a1e66af58b51998, role_id=8051174e7a7947d5b991ebd793c2e543)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:32:59.368914 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b26aff30-9d6c-46c1-8344-4e2ecf44b326 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:32:59.391584 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1068/2142] 10.222.0.49 () {60 vars in 1482 bytes} [Wed Jun 15 10:32:59 2022] PUT /identity/v3/projects/5c52d48547364b2f9bad7234a6189dab/users/6d656a8163e24d538a1e66af58b51998/roles/8051174e7a7947d5b991ebd793c2e543 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:32:59.395580 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ddd2b41-d0d6-410f-80cc-1b16a6e5aa4b admin admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:32:59.406835 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ddd2b41-d0d6-410f-80cc-1b16a6e5aa4b admin admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:32:59.449283 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:DELETE=4,keystone:SELECT=92,keystone:INSERT=4 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:32:59.469648 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea10f9cb-ee6d-4d3a-91b4-60d67deb50c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:32:59.470324 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea10f9cb-ee6d-4d3a-91b4-60d67deb50c2 admin admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:32:59.470605 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea10f9cb-ee6d-4d3a-91b4-60d67deb50c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:32:59.470874 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea10f9cb-ee6d-4d3a-91b4-60d67deb50c2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:32:59.471631 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea10f9cb-ee6d-4d3a-91b4-60d67deb50c2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:32:59.473229 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea10f9cb-ee6d-4d3a-91b4-60d67deb50c2 admin admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:32:59.483663 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1075/2143] 10.222.0.49 () {58 vars in 1121 bytes} [Wed Jun 15 10:32:59 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 10:32:59.487887 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26aff30-9d6c-46c1-8344-4e2ecf44b326 admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:32:59.501308 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26aff30-9d6c-46c1-8344-4e2ecf44b326 admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:32:59.506084 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615482d5-a07d-4988-b598-d04e0ae24ee0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:32:59.507115 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615482d5-a07d-4988-b598-d04e0ae24ee0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:32:59.507416 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615482d5-a07d-4988-b598-d04e0ae24ee0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:32:59.507768 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615482d5-a07d-4988-b598-d04e0ae24ee0 admin admin] PATH_INFO: `/v3/projects/5c52d48547364b2f9bad7234a6189dab/users/6d656a8163e24d538a1e66af58b51998/roles/8051174e7a7947d5b991ebd793c2e543` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:32:59.510255 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-615482d5-a07d-4988-b598-d04e0ae24ee0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5c52d48547364b2f9bad7234a6189dab, user_id=6d656a8163e24d538a1e66af58b51998, role_id=8051174e7a7947d5b991ebd793c2e543)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:32:59.512946 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-615482d5-a07d-4988-b598-d04e0ae24ee0 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:32:59.523750 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1069/2144] 10.222.0.49 () {60 vars in 1482 bytes} [Wed Jun 15 10:32:59 2022] PUT /identity/v3/projects/5c52d48547364b2f9bad7234a6189dab/users/6d656a8163e24d538a1e66af58b51998/roles/8051174e7a7947d5b991ebd793c2e543 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:32:59.535618 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46027fba-1ba4-47d6-bd09-bad0157be56c None None] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:32:59.535907 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46027fba-1ba4-47d6-bd09-bad0157be56c None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:32:59.536188 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46027fba-1ba4-47d6-bd09-bad0157be56c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:32:59.562777 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.auth.core [None req-46027fba-1ba4-47d6-bd09-bad0157be56c None None] MFA Rules not processed for user `6d656a8163e24d538a1e66af58b51998`. Rule list: `[]` (Enabled: `True`). {{(pid=80119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:32:59.564041 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-46027fba-1ba4-47d6-bd09-bad0157be56c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:32:59.634139 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1076/2145] 10.222.0.49 () {58 vars in 957 bytes} [Wed Jun 15 10:32:59 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:32:59.644688 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615482d5-a07d-4988-b598-d04e0ae24ee0 admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:32:59.646424 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-615482d5-a07d-4988-b598-d04e0ae24ee0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:32:59.718322 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615482d5-a07d-4988-b598-d04e0ae24ee0 admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:32:59.722412 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4cb5f81-4f9e-4aac-b416-90c8641e2371 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20d1327553404b80aecee87c78fcb52b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '559b3e713f5844b6ae37151d56dd419f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:32:59.722983 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cb5f81-4f9e-4aac-b416-90c8641e2371 service cinder] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:32:59.723401 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cb5f81-4f9e-4aac-b416-90c8641e2371 service cinder] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:32:59.723588 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cb5f81-4f9e-4aac-b416-90c8641e2371 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:32:59.733307 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4cb5f81-4f9e-4aac-b416-90c8641e2371 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:32:59.734776 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4cb5f81-4f9e-4aac-b416-90c8641e2371 service cinder] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:32:59.746458 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1070/2146] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Jun 15 10:32:59 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:33:00.503466 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46027fba-1ba4-47d6-bd09-bad0157be56c None None] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:33:00.597412 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46027fba-1ba4-47d6-bd09-bad0157be56c None None] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:33:00.606766 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c156be-28c1-429a-b35a-a51727dd4933 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:33:00.608064 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c156be-28c1-429a-b35a-a51727dd4933 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:33:00.611303 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c156be-28c1-429a-b35a-a51727dd4933 admin admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:33:00.611663 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c156be-28c1-429a-b35a-a51727dd4933 admin admin] PATH_INFO: `/v3/users/5aba4ac9eed84879b00c0eea0cc39c72` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:33:00.613608 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c156be-28c1-429a-b35a-a51727dd4933 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5aba4ac9eed84879b00c0eea0cc39c72)` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:33:00.615605 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c156be-28c1-429a-b35a-a51727dd4933 admin admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:33:00.681384 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-45c156be-28c1-429a-b35a-a51727dd4933 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5aba4ac9eed84879b00c0eea0cc39c72'} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:33:00.691918 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-45c156be-28c1-429a-b35a-a51727dd4933 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5aba4ac9eed84879b00c0eea0cc39c72'} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:33:00.692783 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-45c156be-28c1-429a-b35a-a51727dd4933 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5aba4ac9eed84879b00c0eea0cc39c72'} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:33:00.693515 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-45c156be-28c1-429a-b35a-a51727dd4933 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5aba4ac9eed84879b00c0eea0cc39c72'} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:33:00.706219 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-45c156be-28c1-429a-b35a-a51727dd4933 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5aba4ac9eed84879b00c0eea0cc39c72'} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:33:00.712386 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:33:00.730172 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1077/2147] 10.222.0.49 () {58 vars in 1223 bytes} [Wed Jun 15 10:33:00 2022] DELETE /identity/v3/users/5aba4ac9eed84879b00c0eea0cc39c72 => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:33:00.735415 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f12c051-2721-4007-8603-183cff45df63 None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:33:00.736420 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f12c051-2721-4007-8603-183cff45df63 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:33:00.736893 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f12c051-2721-4007-8603-183cff45df63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:33:00.761507 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-3f12c051-2721-4007-8603-183cff45df63 None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:33:00.762772 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-3f12c051-2721-4007-8603-183cff45df63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:33:00.840119 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1071/2148] 10.222.0.49 () {58 vars in 957 bytes} [Wed Jun 15 10:33:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:33:00.850592 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c156be-28c1-429a-b35a-a51727dd4933 admin admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:33:00.852411 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-45c156be-28c1-429a-b35a-a51727dd4933 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:33:00.937450 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c156be-28c1-429a-b35a-a51727dd4933 admin admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:33:00.941954 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8653acd-7fc4-4f85-910e-ae6b03512a1e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0cb44a17d205431793e9a8791e477a32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '559b3e713f5844b6ae37151d56dd419f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:33:00.942625 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8653acd-7fc4-4f85-910e-ae6b03512a1e service neutron] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:33:00.942911 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8653acd-7fc4-4f85-910e-ae6b03512a1e service neutron] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:33:00.943230 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8653acd-7fc4-4f85-910e-ae6b03512a1e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:33:00.955652 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8653acd-7fc4-4f85-910e-ae6b03512a1e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:33:00.957448 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8653acd-7fc4-4f85-910e-ae6b03512a1e service neutron] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:33:00.968836 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1078/2149] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Jun 15 10:33:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:33:01.233149 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f12c051-2721-4007-8603-183cff45df63 None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:33:01.234664 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-3f12c051-2721-4007-8603-183cff45df63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:33:01.247836 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f12c051-2721-4007-8603-183cff45df63 None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:33:01.252504 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba3c5e1-920a-462a-ad30-b8686c706346 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:33:01.253370 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba3c5e1-920a-462a-ad30-b8686c706346 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:33:01.253647 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba3c5e1-920a-462a-ad30-b8686c706346 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:33:01.253925 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba3c5e1-920a-462a-ad30-b8686c706346 admin admin] PATH_INFO: `/v3/projects/a36746ae425246beafadf564a07d8648` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:33:01.255259 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba3c5e1-920a-462a-ad30-b8686c706346 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a36746ae425246beafadf564a07d8648)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:33:01.257163 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba3c5e1-920a-462a-ad30-b8686c706346 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:33:01.303209 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-1ba3c5e1-920a-462a-ad30-b8686c706346 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a36746ae425246beafadf564a07d8648', 'request_id': 'req-1ba3c5e1-920a-462a-ad30-b8686c706346'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:33:01.311186 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-1ba3c5e1-920a-462a-ad30-b8686c706346 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a36746ae425246beafadf564a07d8648', 'request_id': 'req-1ba3c5e1-920a-462a-ad30-b8686c706346'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:33:01.316401 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=91,keystone:INSERT=3,keystone:DELETE=6 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:33:01.318777 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-1ba3c5e1-920a-462a-ad30-b8686c706346 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a36746ae425246beafadf564a07d8648', 'request_id': 'req-1ba3c5e1-920a-462a-ad30-b8686c706346'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:33:01.335262 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-1ba3c5e1-920a-462a-ad30-b8686c706346 admin admin] The token cache is being invalidate because project a36746ae425246beafadf564a07d8648 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80120) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 10:33:01.335547 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-1ba3c5e1-920a-462a-ad30-b8686c706346 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:33:01.336207 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-1ba3c5e1-920a-462a-ad30-b8686c706346 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:33:01.337726 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1072/2150] 10.222.0.49 () {58 vars in 1232 bytes} [Wed Jun 15 10:33:01 2022] DELETE /identity/v3/projects/a36746ae425246beafadf564a07d8648 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:33:01.341633 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8653acd-7fc4-4f85-910e-ae6b03512a1e service neutron] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:33:01.351459 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8653acd-7fc4-4f85-910e-ae6b03512a1e service neutron] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:33:01.406792 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d200e11-1e3a-4dc7-a492-fc73f9967a21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:33:01.407696 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d200e11-1e3a-4dc7-a492-fc73f9967a21 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:33:01.407979 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d200e11-1e3a-4dc7-a492-fc73f9967a21 admin admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:33:01.408249 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d200e11-1e3a-4dc7-a492-fc73f9967a21 admin admin] PATH_INFO: `/v3/users/6d656a8163e24d538a1e66af58b51998` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:33:01.409415 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d200e11-1e3a-4dc7-a492-fc73f9967a21 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6d656a8163e24d538a1e66af58b51998)` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:33:01.411378 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d200e11-1e3a-4dc7-a492-fc73f9967a21 admin admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:33:01.456900 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-7d200e11-1e3a-4dc7-a492-fc73f9967a21 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6d656a8163e24d538a1e66af58b51998'} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:33:01.463619 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-7d200e11-1e3a-4dc7-a492-fc73f9967a21 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6d656a8163e24d538a1e66af58b51998'} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:33:01.464265 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-7d200e11-1e3a-4dc7-a492-fc73f9967a21 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6d656a8163e24d538a1e66af58b51998'} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:33:01.464864 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-7d200e11-1e3a-4dc7-a492-fc73f9967a21 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6d656a8163e24d538a1e66af58b51998'} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:33:01.472094 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-7d200e11-1e3a-4dc7-a492-fc73f9967a21 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6d656a8163e24d538a1e66af58b51998'} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:33:01.493034 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1079/2151] 10.222.0.49 () {58 vars in 1223 bytes} [Wed Jun 15 10:33:01 2022] DELETE /identity/v3/users/6d656a8163e24d538a1e66af58b51998 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:33:01.698984 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba3c5e1-920a-462a-ad30-b8686c706346 admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:33:01.701162 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-1ba3c5e1-920a-462a-ad30-b8686c706346 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:33:01.714314 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba3c5e1-920a-462a-ad30-b8686c706346 admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:33:01.767312 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d200e11-1e3a-4dc7-a492-fc73f9967a21 admin admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:33:01.768804 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-7d200e11-1e3a-4dc7-a492-fc73f9967a21 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:33:01.774827 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65663838-fe5d-4792-a682-3694c34f42e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:33:01.778251 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65663838-fe5d-4792-a682-3694c34f42e4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:33:01.778637 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65663838-fe5d-4792-a682-3694c34f42e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:33:01.778989 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65663838-fe5d-4792-a682-3694c34f42e4 admin admin] PATH_INFO: `/v3/users/139f35f69f4940c69700535d4cd53d50` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:33:01.780431 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65663838-fe5d-4792-a682-3694c34f42e4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=139f35f69f4940c69700535d4cd53d50)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:33:01.782975 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65663838-fe5d-4792-a682-3694c34f42e4 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:33:01.794942 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d200e11-1e3a-4dc7-a492-fc73f9967a21 admin admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:33:01.801604 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc3a126-6383-47cf-be34-2db92b18d68e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:33:01.802611 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc3a126-6383-47cf-be34-2db92b18d68e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:33:01.803116 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc3a126-6383-47cf-be34-2db92b18d68e admin admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:33:01.803511 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc3a126-6383-47cf-be34-2db92b18d68e admin admin] PATH_INFO: `/v3/projects/5c52d48547364b2f9bad7234a6189dab` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:33:01.805013 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc3a126-6383-47cf-be34-2db92b18d68e admin admin] RBAC: Authorizing `identity:delete_project(project_id=5c52d48547364b2f9bad7234a6189dab)` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:33:01.806992 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc3a126-6383-47cf-be34-2db92b18d68e admin admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:33:01.808755 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:DELETE=12,keystone:SELECT=87,keystone:INSERT=1 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:33:01.835381 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-65663838-fe5d-4792-a682-3694c34f42e4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '139f35f69f4940c69700535d4cd53d50'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:33:01.843432 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-65663838-fe5d-4792-a682-3694c34f42e4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '139f35f69f4940c69700535d4cd53d50'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:33:01.844116 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-65663838-fe5d-4792-a682-3694c34f42e4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '139f35f69f4940c69700535d4cd53d50'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:33:01.844715 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-65663838-fe5d-4792-a682-3694c34f42e4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '139f35f69f4940c69700535d4cd53d50'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:33:01.852894 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-65663838-fe5d-4792-a682-3694c34f42e4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '139f35f69f4940c69700535d4cd53d50'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:33:01.859786 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-0dc3a126-6383-47cf-be34-2db92b18d68e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5c52d48547364b2f9bad7234a6189dab', 'request_id': 'req-0dc3a126-6383-47cf-be34-2db92b18d68e'} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:33:01.867435 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-0dc3a126-6383-47cf-be34-2db92b18d68e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5c52d48547364b2f9bad7234a6189dab', 'request_id': 'req-0dc3a126-6383-47cf-be34-2db92b18d68e'} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:33:01.871061 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1073/2152] 10.222.0.49 () {58 vars in 1223 bytes} [Wed Jun 15 10:33:01 2022] DELETE /identity/v3/users/139f35f69f4940c69700535d4cd53d50 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:33:01.873594 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-0dc3a126-6383-47cf-be34-2db92b18d68e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5c52d48547364b2f9bad7234a6189dab', 'request_id': 'req-0dc3a126-6383-47cf-be34-2db92b18d68e'} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:33:01.876120 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f165a0-f133-40e0-bd26-65f817830173 None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:33:01.876497 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f165a0-f133-40e0-bd26-65f817830173 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:33:01.876812 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f165a0-f133-40e0-bd26-65f817830173 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:33:01.887758 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-0dc3a126-6383-47cf-be34-2db92b18d68e admin admin] The token cache is being invalidate because project 5c52d48547364b2f9bad7234a6189dab was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80119) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 10:33:01.888189 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-0dc3a126-6383-47cf-be34-2db92b18d68e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:33:01.889004 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-0dc3a126-6383-47cf-be34-2db92b18d68e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:33:01.890687 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1080/2153] 10.222.0.49 () {58 vars in 1232 bytes} [Wed Jun 15 10:33:01 2022] DELETE /identity/v3/projects/5c52d48547364b2f9bad7234a6189dab => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 10:33:01.896666 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d225865a-fdf7-40bc-b8cb-86430af584b6 None None] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:33:01.897039 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d225865a-fdf7-40bc-b8cb-86430af584b6 None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:33:01.897444 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d225865a-fdf7-40bc-b8cb-86430af584b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:33:01.898050 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-55f165a0-f133-40e0-bd26-65f817830173 None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:33:01.899972 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-55f165a0-f133-40e0-bd26-65f817830173 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:33:01.918397 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.auth.core [None req-d225865a-fdf7-40bc-b8cb-86430af584b6 None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:33:01.919642 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-d225865a-fdf7-40bc-b8cb-86430af584b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:33:01.967893 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1074/2154] 10.222.0.49 () {58 vars in 957 bytes} [Wed Jun 15 10:33:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 10:33:01.974553 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f165a0-f133-40e0-bd26-65f817830173 None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:33:01.976003 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-55f165a0-f133-40e0-bd26-65f817830173 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:33:01.990233 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:DELETE=11,keystone:SELECT=88,keystone:INSERT=1 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:33:01.997426 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1081/2155] 10.222.0.49 () {58 vars in 957 bytes} [Wed Jun 15 10:33:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 10:33:02.005279 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d060d44d-615b-4e52-a036-3bfce84eb3f1 None None] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:33:02.005740 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d060d44d-615b-4e52-a036-3bfce84eb3f1 None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:33:02.006028 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d060d44d-615b-4e52-a036-3bfce84eb3f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:33:02.026892 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.auth.core [None req-d060d44d-615b-4e52-a036-3bfce84eb3f1 None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:33:02.027916 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-d060d44d-615b-4e52-a036-3bfce84eb3f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:33:02.042787 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1082/2156] 10.222.0.49 () {58 vars in 957 bytes} [Wed Jun 15 10:33:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 10:33:02.044008 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d060d44d-615b-4e52-a036-3bfce84eb3f1 None None] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:33:02.048434 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f165a0-f133-40e0-bd26-65f817830173 None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:33:02.052740 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ec682f-9d8c-473f-842f-ad1718695473 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0cb44a17d205431793e9a8791e477a32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '559b3e713f5844b6ae37151d56dd419f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:33:02.052954 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ec682f-9d8c-473f-842f-ad1718695473 service neutron] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:33:02.053216 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d060d44d-615b-4e52-a036-3bfce84eb3f1 None None] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:33:02.053434 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ec682f-9d8c-473f-842f-ad1718695473 service neutron] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:33:02.053785 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ec682f-9d8c-473f-842f-ad1718695473 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:33:02.058384 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abcf0d08-b930-41a7-abe2-82a21ec5d713 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:33:02.059824 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcf0d08-b930-41a7-abe2-82a21ec5d713 admin admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:33:02.060562 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcf0d08-b930-41a7-abe2-82a21ec5d713 admin admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:33:02.060850 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcf0d08-b930-41a7-abe2-82a21ec5d713 admin admin] PATH_INFO: `/v3/domains` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:33:02.062487 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abcf0d08-b930-41a7-abe2-82a21ec5d713 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:33:02.063645 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abcf0d08-b930-41a7-abe2-82a21ec5d713 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:33:02.063748 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ec682f-9d8c-473f-842f-ad1718695473 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:33:02.065103 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ec682f-9d8c-473f-842f-ad1718695473 service neutron] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:33:02.065290 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abcf0d08-b930-41a7-abe2-82a21ec5d713 admin admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:33:02.075559 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1075/2157] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Jun 15 10:33:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:33:02.082751 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1083/2158] 10.222.0.49 () {58 vars in 1152 bytes} [Wed Jun 15 10:33:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:33:02.083803 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ec682f-9d8c-473f-842f-ad1718695473 service neutron] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:33:02.094714 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ec682f-9d8c-473f-842f-ad1718695473 service neutron] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:33:02.099514 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4bc606-bf1a-4f1b-8e99-f4befc4e54db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:33:02.100179 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4bc606-bf1a-4f1b-8e99-f4befc4e54db admin admin] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:33:02.100466 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4bc606-bf1a-4f1b-8e99-f4befc4e54db admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:33:02.100871 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4bc606-bf1a-4f1b-8e99-f4befc4e54db admin admin] PATH_INFO: `/v3/projects` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:33:02.102312 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d4bc606-bf1a-4f1b-8e99-f4befc4e54db admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:33:02.104265 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d4bc606-bf1a-4f1b-8e99-f4befc4e54db admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:33:02.135381 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1076/2159] 10.222.0.49 () {60 vars in 1152 bytes} [Wed Jun 15 10:33:02 2022] POST /identity/v3/projects => generated 391 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 10:33:02.139542 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abcf0d08-b930-41a7-abe2-82a21ec5d713 admin admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:33:02.153439 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abcf0d08-b930-41a7-abe2-82a21ec5d713 admin admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:33:02.221839 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dfb30ff-3287-4260-9143-8a33f6138079 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:33:02.222888 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfb30ff-3287-4260-9143-8a33f6138079 admin admin] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:33:02.223314 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfb30ff-3287-4260-9143-8a33f6138079 admin admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:33:02.223716 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfb30ff-3287-4260-9143-8a33f6138079 admin admin] PATH_INFO: `/v3/users` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:33:02.224476 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dfb30ff-3287-4260-9143-8a33f6138079 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:33:02.226532 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dfb30ff-3287-4260-9143-8a33f6138079 admin admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:33:02.251459 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:33:02.259588 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1084/2160] 10.222.0.49 () {60 vars in 1143 bytes} [Wed Jun 15 10:33:02 2022] POST /identity/v3/users => generated 296 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 10:33:02.264032 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4bc606-bf1a-4f1b-8e99-f4befc4e54db admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:33:02.277733 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4bc606-bf1a-4f1b-8e99-f4befc4e54db admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:33:02.284043 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22332b8-359a-419f-9838-63f9067145cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:33:02.284708 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22332b8-359a-419f-9838-63f9067145cf admin admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:33:02.285096 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22332b8-359a-419f-9838-63f9067145cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:33:02.285479 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22332b8-359a-419f-9838-63f9067145cf admin admin] PATH_INFO: `/v3/roles` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:33:02.286324 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f22332b8-359a-419f-9838-63f9067145cf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:33:02.288095 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f22332b8-359a-419f-9838-63f9067145cf admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:33:02.300606 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1077/2161] 10.222.0.49 () {58 vars in 1121 bytes} [Wed Jun 15 10:33:02 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 10:33:02.304637 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dfb30ff-3287-4260-9143-8a33f6138079 admin admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:33:02.316359 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dfb30ff-3287-4260-9143-8a33f6138079 admin admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:33:02.322361 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff29205-6347-4e63-88cb-d32977fbcd3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:33:02.323374 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff29205-6347-4e63-88cb-d32977fbcd3c admin admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:33:02.323788 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff29205-6347-4e63-88cb-d32977fbcd3c admin admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:33:02.324713 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff29205-6347-4e63-88cb-d32977fbcd3c admin admin] PATH_INFO: `/v3/roles` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:33:02.325586 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff29205-6347-4e63-88cb-d32977fbcd3c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:33:02.327349 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff29205-6347-4e63-88cb-d32977fbcd3c admin admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:33:02.341345 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1085/2162] 10.222.0.49 () {58 vars in 1121 bytes} [Wed Jun 15 10:33:02 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 10:33:02.345955 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22332b8-359a-419f-9838-63f9067145cf admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:33:02.356934 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22332b8-359a-419f-9838-63f9067145cf admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:33:02.365380 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5a0b70-3aaa-45c9-a63f-14643edef464 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:33:02.366388 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5a0b70-3aaa-45c9-a63f-14643edef464 admin admin] REQUEST_METHOD: `PUT` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:33:02.366776 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5a0b70-3aaa-45c9-a63f-14643edef464 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:33:02.368077 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5a0b70-3aaa-45c9-a63f-14643edef464 admin admin] PATH_INFO: `/v3/projects/c83fd32c69114456a9a01cceaadb6686/users/cb44dcb7cba44464ad9f731b41269802/roles/6876e57cc51c448097ab109d35b2a3da` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:33:02.383794 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5a0b70-3aaa-45c9-a63f-14643edef464 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c83fd32c69114456a9a01cceaadb6686, user_id=cb44dcb7cba44464ad9f731b41269802, role_id=6876e57cc51c448097ab109d35b2a3da)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:33:02.386433 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5a0b70-3aaa-45c9-a63f-14643edef464 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:33:02.402431 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1078/2163] 10.222.0.49 () {60 vars in 1482 bytes} [Wed Jun 15 10:33:02 2022] PUT /identity/v3/projects/c83fd32c69114456a9a01cceaadb6686/users/cb44dcb7cba44464ad9f731b41269802/roles/6876e57cc51c448097ab109d35b2a3da => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:33:02.412870 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e363c4dc-1b6d-4df2-bec8-17507cb68219 None None] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:33:02.413166 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e363c4dc-1b6d-4df2-bec8-17507cb68219 None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:33:02.413456 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e363c4dc-1b6d-4df2-bec8-17507cb68219 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:33:02.439910 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.auth.core [None req-e363c4dc-1b6d-4df2-bec8-17507cb68219 None None] MFA Rules not processed for user `cb44dcb7cba44464ad9f731b41269802`. Rule list: `[]` (Enabled: `True`). {{(pid=80119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:33:02.441146 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-e363c4dc-1b6d-4df2-bec8-17507cb68219 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:33:02.512323 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1086/2164] 10.222.0.49 () {58 vars in 957 bytes} [Wed Jun 15 10:33:02 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:33:02.522749 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5a0b70-3aaa-45c9-a63f-14643edef464 admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:33:02.524501 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-ce5a0b70-3aaa-45c9-a63f-14643edef464 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:33:02.539967 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e363c4dc-1b6d-4df2-bec8-17507cb68219 None None] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:33:02.541427 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-e363c4dc-1b6d-4df2-bec8-17507cb68219 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:33:02.631283 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5a0b70-3aaa-45c9-a63f-14643edef464 admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:33:02.635842 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a488c16-f90d-4c83-a827-dad2bd0303dc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20d1327553404b80aecee87c78fcb52b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '559b3e713f5844b6ae37151d56dd419f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:33:02.636566 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a488c16-f90d-4c83-a827-dad2bd0303dc service cinder] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:33:02.636918 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a488c16-f90d-4c83-a827-dad2bd0303dc service cinder] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:33:02.637264 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a488c16-f90d-4c83-a827-dad2bd0303dc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:33:02.644996 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e363c4dc-1b6d-4df2-bec8-17507cb68219 None None] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:33:02.647593 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a488c16-f90d-4c83-a827-dad2bd0303dc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:33:02.649061 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a488c16-f90d-4c83-a827-dad2bd0303dc service cinder] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:33:02.649917 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8e2fce-1122-4d06-b9fd-d53ca0a8dbd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:33:02.650789 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8e2fce-1122-4d06-b9fd-d53ca0a8dbd6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:33:02.651076 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8e2fce-1122-4d06-b9fd-d53ca0a8dbd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:33:02.651458 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8e2fce-1122-4d06-b9fd-d53ca0a8dbd6 admin admin] PATH_INFO: `/v3/projects/722b2275169048cc8263fd39ac8c0959` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:33:02.652985 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff8e2fce-1122-4d06-b9fd-d53ca0a8dbd6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=722b2275169048cc8263fd39ac8c0959)` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:33:02.654594 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff8e2fce-1122-4d06-b9fd-d53ca0a8dbd6 admin admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:33:02.658574 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1079/2165] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Jun 15 10:33:02 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:33:02.709167 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-ff8e2fce-1122-4d06-b9fd-d53ca0a8dbd6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '722b2275169048cc8263fd39ac8c0959', 'request_id': 'req-ff8e2fce-1122-4d06-b9fd-d53ca0a8dbd6'} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:33:02.717606 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-ff8e2fce-1122-4d06-b9fd-d53ca0a8dbd6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '722b2275169048cc8263fd39ac8c0959', 'request_id': 'req-ff8e2fce-1122-4d06-b9fd-d53ca0a8dbd6'} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:33:02.724836 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-ff8e2fce-1122-4d06-b9fd-d53ca0a8dbd6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '722b2275169048cc8263fd39ac8c0959', 'request_id': 'req-ff8e2fce-1122-4d06-b9fd-d53ca0a8dbd6'} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:33:02.740930 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-ff8e2fce-1122-4d06-b9fd-d53ca0a8dbd6 admin admin] The token cache is being invalidate because project 722b2275169048cc8263fd39ac8c0959 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80119) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 10:33:02.741283 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-ff8e2fce-1122-4d06-b9fd-d53ca0a8dbd6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:33:02.742967 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-ff8e2fce-1122-4d06-b9fd-d53ca0a8dbd6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:33:02.745021 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1087/2166] 10.222.0.49 () {58 vars in 1232 bytes} [Wed Jun 15 10:33:02 2022] DELETE /identity/v3/projects/722b2275169048cc8263fd39ac8c0959 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 10:33:02.748755 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a488c16-f90d-4c83-a827-dad2bd0303dc service cinder] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:33:02.761694 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a488c16-f90d-4c83-a827-dad2bd0303dc service cinder] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:33:02.839396 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:33:02.845899 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6569400-462d-4e09-a566-ab77fbe45496 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:33:02.847001 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6569400-462d-4e09-a566-ab77fbe45496 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:33:02.847648 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6569400-462d-4e09-a566-ab77fbe45496 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:33:02.847936 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6569400-462d-4e09-a566-ab77fbe45496 admin admin] PATH_INFO: `/v3/users/d441c82018634d9796732339914a8559` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:33:02.849280 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6569400-462d-4e09-a566-ab77fbe45496 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d441c82018634d9796732339914a8559)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:33:02.852781 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6569400-462d-4e09-a566-ab77fbe45496 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:33:02.906828 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-c6569400-462d-4e09-a566-ab77fbe45496 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd441c82018634d9796732339914a8559'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:33:02.915536 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-c6569400-462d-4e09-a566-ab77fbe45496 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd441c82018634d9796732339914a8559'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:33:02.916923 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-c6569400-462d-4e09-a566-ab77fbe45496 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd441c82018634d9796732339914a8559'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:33:02.917569 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-c6569400-462d-4e09-a566-ab77fbe45496 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd441c82018634d9796732339914a8559'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:33:02.926313 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-c6569400-462d-4e09-a566-ab77fbe45496 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd441c82018634d9796732339914a8559'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:33:02.956542 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1080/2167] 10.222.0.49 () {58 vars in 1223 bytes} [Wed Jun 15 10:33:02 2022] DELETE /identity/v3/users/d441c82018634d9796732339914a8559 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 10:33:03.230998 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8e2fce-1122-4d06-b9fd-d53ca0a8dbd6 admin admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:33:03.232527 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-ff8e2fce-1122-4d06-b9fd-d53ca0a8dbd6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:33:03.246613 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8e2fce-1122-4d06-b9fd-d53ca0a8dbd6 admin admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:33:03.251489 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:33:03.306080 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2b6553-38bb-473e-befe-2e642174accd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:33:03.306949 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2b6553-38bb-473e-befe-2e642174accd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:33:03.307275 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2b6553-38bb-473e-befe-2e642174accd admin admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:33:03.307552 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2b6553-38bb-473e-befe-2e642174accd admin admin] PATH_INFO: `/v3/projects/13c11364ff594f9597521c3e6f9c03bd` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:33:03.308793 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd2b6553-38bb-473e-befe-2e642174accd admin admin] RBAC: Authorizing `identity:delete_project(project_id=13c11364ff594f9597521c3e6f9c03bd)` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:33:03.310725 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd2b6553-38bb-473e-befe-2e642174accd admin admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:33:03.357000 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-bd2b6553-38bb-473e-befe-2e642174accd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '13c11364ff594f9597521c3e6f9c03bd', 'request_id': 'req-bd2b6553-38bb-473e-befe-2e642174accd'} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:33:03.363906 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-bd2b6553-38bb-473e-befe-2e642174accd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '13c11364ff594f9597521c3e6f9c03bd', 'request_id': 'req-bd2b6553-38bb-473e-befe-2e642174accd'} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:33:03.369346 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-bd2b6553-38bb-473e-befe-2e642174accd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '13c11364ff594f9597521c3e6f9c03bd', 'request_id': 'req-bd2b6553-38bb-473e-befe-2e642174accd'} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:33:03.380243 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-bd2b6553-38bb-473e-befe-2e642174accd admin admin] The token cache is being invalidate because project 13c11364ff594f9597521c3e6f9c03bd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80119) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 10:33:03.380544 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-bd2b6553-38bb-473e-befe-2e642174accd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:33:03.381172 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-bd2b6553-38bb-473e-befe-2e642174accd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:33:03.382812 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1088/2168] 10.222.0.49 () {58 vars in 1232 bytes} [Wed Jun 15 10:33:03 2022] DELETE /identity/v3/projects/13c11364ff594f9597521c3e6f9c03bd => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:33:03.386630 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6569400-462d-4e09-a566-ab77fbe45496 admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:33:03.388031 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-c6569400-462d-4e09-a566-ab77fbe45496 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:33:03.401332 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6569400-462d-4e09-a566-ab77fbe45496 admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:33:03.461274 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2aa88e-c3b4-4bd9-9b2e-fe75cdaf09f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:33:03.462278 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2aa88e-c3b4-4bd9-9b2e-fe75cdaf09f4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:33:03.462561 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2aa88e-c3b4-4bd9-9b2e-fe75cdaf09f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:33:03.462832 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2aa88e-c3b4-4bd9-9b2e-fe75cdaf09f4 admin admin] PATH_INFO: `/v3/users/1be0f3a7b0c74b0fbbd0ed6c401681bf` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:33:03.464074 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f2aa88e-c3b4-4bd9-9b2e-fe75cdaf09f4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1be0f3a7b0c74b0fbbd0ed6c401681bf)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:33:03.465976 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f2aa88e-c3b4-4bd9-9b2e-fe75cdaf09f4 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:33:03.514186 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-8f2aa88e-c3b4-4bd9-9b2e-fe75cdaf09f4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1be0f3a7b0c74b0fbbd0ed6c401681bf'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:33:03.521604 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-8f2aa88e-c3b4-4bd9-9b2e-fe75cdaf09f4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1be0f3a7b0c74b0fbbd0ed6c401681bf'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:33:03.522360 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-8f2aa88e-c3b4-4bd9-9b2e-fe75cdaf09f4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1be0f3a7b0c74b0fbbd0ed6c401681bf'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:33:03.522960 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-8f2aa88e-c3b4-4bd9-9b2e-fe75cdaf09f4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1be0f3a7b0c74b0fbbd0ed6c401681bf'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:33:03.532863 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-8f2aa88e-c3b4-4bd9-9b2e-fe75cdaf09f4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1be0f3a7b0c74b0fbbd0ed6c401681bf'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:33:03.545406 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:33:03.553116 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1081/2169] 10.222.0.49 () {58 vars in 1223 bytes} [Wed Jun 15 10:33:03 2022] DELETE /identity/v3/users/1be0f3a7b0c74b0fbbd0ed6c401681bf => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:33:03.823854 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2b6553-38bb-473e-befe-2e642174accd admin admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:33:03.825494 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-bd2b6553-38bb-473e-befe-2e642174accd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:33:03.840766 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2b6553-38bb-473e-befe-2e642174accd admin admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:33:03.898849 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68ca79b-ade3-400f-a260-a1a6537d487b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:33:03.899691 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68ca79b-ade3-400f-a260-a1a6537d487b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:33:03.899978 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68ca79b-ade3-400f-a260-a1a6537d487b admin admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:33:03.900257 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68ca79b-ade3-400f-a260-a1a6537d487b admin admin] PATH_INFO: `/v3/projects/28ef052d0a624e69ac7e8798ef76235e` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:33:03.901441 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a68ca79b-ade3-400f-a260-a1a6537d487b admin admin] RBAC: Authorizing `identity:delete_project(project_id=28ef052d0a624e69ac7e8798ef76235e)` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:33:03.903386 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a68ca79b-ade3-400f-a260-a1a6537d487b admin admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:33:03.942428 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:33:03.949853 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-a68ca79b-ade3-400f-a260-a1a6537d487b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '28ef052d0a624e69ac7e8798ef76235e', 'request_id': 'req-a68ca79b-ade3-400f-a260-a1a6537d487b'} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:33:03.957727 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-a68ca79b-ade3-400f-a260-a1a6537d487b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '28ef052d0a624e69ac7e8798ef76235e', 'request_id': 'req-a68ca79b-ade3-400f-a260-a1a6537d487b'} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:33:03.963871 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-a68ca79b-ade3-400f-a260-a1a6537d487b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '28ef052d0a624e69ac7e8798ef76235e', 'request_id': 'req-a68ca79b-ade3-400f-a260-a1a6537d487b'} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:33:03.974646 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-a68ca79b-ade3-400f-a260-a1a6537d487b admin admin] The token cache is being invalidate because project 28ef052d0a624e69ac7e8798ef76235e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80119) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 10:33:03.974932 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-a68ca79b-ade3-400f-a260-a1a6537d487b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:33:03.975592 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-a68ca79b-ade3-400f-a260-a1a6537d487b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:33:03.977131 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1089/2170] 10.222.0.49 () {58 vars in 1232 bytes} [Wed Jun 15 10:33:03 2022] DELETE /identity/v3/projects/28ef052d0a624e69ac7e8798ef76235e => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:33:03.981928 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7cba64-d387-4e11-bd85-a2efd0d5abab None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:33:03.982433 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7cba64-d387-4e11-bd85-a2efd0d5abab None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:33:03.982535 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7cba64-d387-4e11-bd85-a2efd0d5abab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:33:04.000329 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-af7cba64-d387-4e11-bd85-a2efd0d5abab None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:33:04.001543 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-af7cba64-d387-4e11-bd85-a2efd0d5abab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:33:04.063697 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1082/2171] 10.222.0.49 () {58 vars in 957 bytes} [Wed Jun 15 10:33:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:33:04.072950 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7e2138-c589-4117-9838-eb8ae4e8ebf2 None None] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:33:04.073240 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7e2138-c589-4117-9838-eb8ae4e8ebf2 None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:33:04.073515 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7e2138-c589-4117-9838-eb8ae4e8ebf2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:33:04.093298 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.auth.core [None req-cf7e2138-c589-4117-9838-eb8ae4e8ebf2 None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:33:04.094495 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-cf7e2138-c589-4117-9838-eb8ae4e8ebf2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:33:04.102627 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1090/2172] 10.222.0.49 () {58 vars in 957 bytes} [Wed Jun 15 10:33:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:33:04.107469 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7cba64-d387-4e11-bd85-a2efd0d5abab None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:33:04.122754 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7cba64-d387-4e11-bd85-a2efd0d5abab None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:33:04.128122 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f195d88e-f7cb-4f30-b1b1-c5b7bc86c3d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:33:04.128862 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f195d88e-f7cb-4f30-b1b1-c5b7bc86c3d8 admin admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:33:04.129195 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f195d88e-f7cb-4f30-b1b1-c5b7bc86c3d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:33:04.129503 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f195d88e-f7cb-4f30-b1b1-c5b7bc86c3d8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:33:04.130376 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f195d88e-f7cb-4f30-b1b1-c5b7bc86c3d8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:33:04.130669 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f195d88e-f7cb-4f30-b1b1-c5b7bc86c3d8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:33:04.132535 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f195d88e-f7cb-4f30-b1b1-c5b7bc86c3d8 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:33:04.145596 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1083/2173] 10.222.0.49 () {58 vars in 1152 bytes} [Wed Jun 15 10:33:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 10:33:04.150209 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7e2138-c589-4117-9838-eb8ae4e8ebf2 None None] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:33:04.159471 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7e2138-c589-4117-9838-eb8ae4e8ebf2 None None] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:33:04.164149 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f4e7bd-7a44-4f65-af62-f070e213a430 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:33:04.164805 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f4e7bd-7a44-4f65-af62-f070e213a430 admin admin] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:33:04.165085 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f4e7bd-7a44-4f65-af62-f070e213a430 admin admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:33:04.165360 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f4e7bd-7a44-4f65-af62-f070e213a430 admin admin] PATH_INFO: `/v3/projects` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:33:04.166032 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f4e7bd-7a44-4f65-af62-f070e213a430 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:33:04.167962 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f4e7bd-7a44-4f65-af62-f070e213a430 admin admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:33:04.194783 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1091/2174] 10.222.0.49 () {60 vars in 1152 bytes} [Wed Jun 15 10:33:04 2022] POST /identity/v3/projects => generated 385 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 10:33:04.199493 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f195d88e-f7cb-4f30-b1b1-c5b7bc86c3d8 admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:33:04.213948 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f195d88e-f7cb-4f30-b1b1-c5b7bc86c3d8 admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:33:04.273132 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6d29bc-3fc2-443e-8d57-a03c7553f9a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:33:04.273795 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6d29bc-3fc2-443e-8d57-a03c7553f9a4 admin admin] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:33:04.274073 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6d29bc-3fc2-443e-8d57-a03c7553f9a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:33:04.274344 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6d29bc-3fc2-443e-8d57-a03c7553f9a4 admin admin] PATH_INFO: `/v3/users` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:33:04.275036 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6d29bc-3fc2-443e-8d57-a03c7553f9a4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:33:04.276958 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6d29bc-3fc2-443e-8d57-a03c7553f9a4 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:33:04.296643 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1084/2175] 10.222.0.49 () {60 vars in 1143 bytes} [Wed Jun 15 10:33:04 2022] POST /identity/v3/users => generated 293 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 10:33:04.300778 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f4e7bd-7a44-4f65-af62-f070e213a430 admin admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:33:04.311776 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f4e7bd-7a44-4f65-af62-f070e213a430 admin admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:33:04.316561 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6fdd2a-a527-4375-b742-e37ba8adf7c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:33:04.317221 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6fdd2a-a527-4375-b742-e37ba8adf7c0 admin admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:33:04.317507 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6fdd2a-a527-4375-b742-e37ba8adf7c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:33:04.317789 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6fdd2a-a527-4375-b742-e37ba8adf7c0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:33:04.318515 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb6fdd2a-a527-4375-b742-e37ba8adf7c0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:33:04.320790 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb6fdd2a-a527-4375-b742-e37ba8adf7c0 admin admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:33:04.330990 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1092/2176] 10.222.0.49 () {58 vars in 1121 bytes} [Wed Jun 15 10:33:04 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 10:33:04.334839 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6d29bc-3fc2-443e-8d57-a03c7553f9a4 admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:33:04.345011 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6d29bc-3fc2-443e-8d57-a03c7553f9a4 admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:33:04.349751 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3a717b-983a-43e8-9436-18d95466ad11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:33:04.350402 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3a717b-983a-43e8-9436-18d95466ad11 admin admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:33:04.350688 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3a717b-983a-43e8-9436-18d95466ad11 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:33:04.350967 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3a717b-983a-43e8-9436-18d95466ad11 admin admin] PATH_INFO: `/v3/roles` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:33:04.351768 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce3a717b-983a-43e8-9436-18d95466ad11 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:33:04.353373 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce3a717b-983a-43e8-9436-18d95466ad11 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:33:04.363112 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1085/2177] 10.222.0.49 () {58 vars in 1121 bytes} [Wed Jun 15 10:33:04 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 10:33:04.367571 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6fdd2a-a527-4375-b742-e37ba8adf7c0 admin admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:33:04.377453 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6fdd2a-a527-4375-b742-e37ba8adf7c0 admin admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:33:04.382111 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4541c627-8994-421f-8772-9784d0c9361c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:33:04.383620 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4541c627-8994-421f-8772-9784d0c9361c admin admin] REQUEST_METHOD: `PUT` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:33:04.383904 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4541c627-8994-421f-8772-9784d0c9361c admin admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:33:04.384207 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4541c627-8994-421f-8772-9784d0c9361c admin admin] PATH_INFO: `/v3/projects/51c380340d5446679239a1103a7ace03/users/3a2a22b0aeec40a2aa02ff5f91a07afa/roles/6876e57cc51c448097ab109d35b2a3da` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:33:04.398649 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4541c627-8994-421f-8772-9784d0c9361c admin admin] RBAC: Authorizing `identity:create_grant(project_id=51c380340d5446679239a1103a7ace03, user_id=3a2a22b0aeec40a2aa02ff5f91a07afa, role_id=6876e57cc51c448097ab109d35b2a3da)` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:33:04.401381 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4541c627-8994-421f-8772-9784d0c9361c admin admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:33:04.415642 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1093/2178] 10.222.0.49 () {60 vars in 1482 bytes} [Wed Jun 15 10:33:04 2022] PUT /identity/v3/projects/51c380340d5446679239a1103a7ace03/users/3a2a22b0aeec40a2aa02ff5f91a07afa/roles/6876e57cc51c448097ab109d35b2a3da => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:33:04.425591 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbaa691-da5c-4a21-a208-a0e6edbb3c9b None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:33:04.425984 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbaa691-da5c-4a21-a208-a0e6edbb3c9b None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:33:04.426329 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbaa691-da5c-4a21-a208-a0e6edbb3c9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:33:04.452820 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-9cbaa691-da5c-4a21-a208-a0e6edbb3c9b None None] MFA Rules not processed for user `3a2a22b0aeec40a2aa02ff5f91a07afa`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:33:04.454035 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-9cbaa691-da5c-4a21-a208-a0e6edbb3c9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:33:04.500498 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:33:04.518324 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1086/2179] 10.222.0.49 () {58 vars in 957 bytes} [Wed Jun 15 10:33:04 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:33:04.529805 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4541c627-8994-421f-8772-9784d0c9361c admin admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:33:04.531821 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-4541c627-8994-421f-8772-9784d0c9361c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:33:04.591268 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4541c627-8994-421f-8772-9784d0c9361c admin admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:33:04.594890 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2991ca-2ec0-4d6a-a31b-a558cb5424f9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20d1327553404b80aecee87c78fcb52b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '559b3e713f5844b6ae37151d56dd419f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:33:04.595567 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2991ca-2ec0-4d6a-a31b-a558cb5424f9 service cinder] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:33:04.595850 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2991ca-2ec0-4d6a-a31b-a558cb5424f9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:33:04.596131 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2991ca-2ec0-4d6a-a31b-a558cb5424f9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:33:04.607549 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c2991ca-2ec0-4d6a-a31b-a558cb5424f9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:33:04.609294 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c2991ca-2ec0-4d6a-a31b-a558cb5424f9 service cinder] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:33:04.620459 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1094/2180] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Jun 15 10:33:04 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:33:13.430945 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbaa691-da5c-4a21-a208-a0e6edbb3c9b None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:33:13.492487 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbaa691-da5c-4a21-a208-a0e6edbb3c9b None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:33:13.497003 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ffb5aa6-60bb-44c5-a18a-d7c3fcef2667 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:33:13.497874 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffb5aa6-60bb-44c5-a18a-d7c3fcef2667 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:33:13.498160 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffb5aa6-60bb-44c5-a18a-d7c3fcef2667 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:33:13.498441 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffb5aa6-60bb-44c5-a18a-d7c3fcef2667 admin admin] PATH_INFO: `/v3/users/3a2a22b0aeec40a2aa02ff5f91a07afa` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:33:13.499692 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ffb5aa6-60bb-44c5-a18a-d7c3fcef2667 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3a2a22b0aeec40a2aa02ff5f91a07afa)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:33:13.501608 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ffb5aa6-60bb-44c5-a18a-d7c3fcef2667 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:33:13.545773 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-7ffb5aa6-60bb-44c5-a18a-d7c3fcef2667 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3a2a22b0aeec40a2aa02ff5f91a07afa'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:33:13.552448 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-7ffb5aa6-60bb-44c5-a18a-d7c3fcef2667 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3a2a22b0aeec40a2aa02ff5f91a07afa'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:33:13.553093 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-7ffb5aa6-60bb-44c5-a18a-d7c3fcef2667 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3a2a22b0aeec40a2aa02ff5f91a07afa'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:33:13.553691 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-7ffb5aa6-60bb-44c5-a18a-d7c3fcef2667 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3a2a22b0aeec40a2aa02ff5f91a07afa'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:33:13.560484 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-7ffb5aa6-60bb-44c5-a18a-d7c3fcef2667 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3a2a22b0aeec40a2aa02ff5f91a07afa'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:33:13.578368 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1087/2181] 10.222.0.49 () {58 vars in 1223 bytes} [Wed Jun 15 10:33:13 2022] DELETE /identity/v3/users/3a2a22b0aeec40a2aa02ff5f91a07afa => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:33:13.583197 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caacd909-8139-464b-8364-4048391c6217 None None] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:33:13.583682 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caacd909-8139-464b-8364-4048391c6217 None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:33:13.583813 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caacd909-8139-464b-8364-4048391c6217 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:33:13.601319 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.auth.core [None req-caacd909-8139-464b-8364-4048391c6217 None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:33:13.602807 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-caacd909-8139-464b-8364-4048391c6217 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:33:13.662838 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1095/2182] 10.222.0.49 () {58 vars in 957 bytes} [Wed Jun 15 10:33:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:33:13.672866 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ffb5aa6-60bb-44c5-a18a-d7c3fcef2667 admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:33:13.674315 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-7ffb5aa6-60bb-44c5-a18a-d7c3fcef2667 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:33:13.731599 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ffb5aa6-60bb-44c5-a18a-d7c3fcef2667 admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:33:13.735331 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28aaa970-f472-4b61-9771-7bd37d8ab07b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0cb44a17d205431793e9a8791e477a32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '559b3e713f5844b6ae37151d56dd419f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:33:13.735982 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28aaa970-f472-4b61-9771-7bd37d8ab07b service neutron] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:33:13.736270 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28aaa970-f472-4b61-9771-7bd37d8ab07b service neutron] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:33:13.736552 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28aaa970-f472-4b61-9771-7bd37d8ab07b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:33:13.746475 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28aaa970-f472-4b61-9771-7bd37d8ab07b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:33:13.747887 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28aaa970-f472-4b61-9771-7bd37d8ab07b service neutron] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:33:13.757799 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1088/2183] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Jun 15 10:33:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:33:14.047988 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caacd909-8139-464b-8364-4048391c6217 None None] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:33:14.050579 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-caacd909-8139-464b-8364-4048391c6217 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:33:14.073478 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caacd909-8139-464b-8364-4048391c6217 None None] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:33:14.088246 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e129da0d-51fe-4430-ad00-6fafc1b130ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:33:14.089403 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e129da0d-51fe-4430-ad00-6fafc1b130ac admin admin] REQUEST_METHOD: `DELETE` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:33:14.089639 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e129da0d-51fe-4430-ad00-6fafc1b130ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:33:14.089940 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e129da0d-51fe-4430-ad00-6fafc1b130ac admin admin] PATH_INFO: `/v3/projects/51c380340d5446679239a1103a7ace03` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:33:14.091556 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e129da0d-51fe-4430-ad00-6fafc1b130ac admin admin] RBAC: Authorizing `identity:delete_project(project_id=51c380340d5446679239a1103a7ace03)` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:33:14.093852 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e129da0d-51fe-4430-ad00-6fafc1b130ac admin admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:33:14.096912 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=93,keystone:DELETE=4,keystone:INSERT=3 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:33:14.151879 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-e129da0d-51fe-4430-ad00-6fafc1b130ac admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '51c380340d5446679239a1103a7ace03', 'request_id': 'req-e129da0d-51fe-4430-ad00-6fafc1b130ac'} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:33:14.161057 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-e129da0d-51fe-4430-ad00-6fafc1b130ac admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '51c380340d5446679239a1103a7ace03', 'request_id': 'req-e129da0d-51fe-4430-ad00-6fafc1b130ac'} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:33:14.167150 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-e129da0d-51fe-4430-ad00-6fafc1b130ac admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '51c380340d5446679239a1103a7ace03', 'request_id': 'req-e129da0d-51fe-4430-ad00-6fafc1b130ac'} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:33:14.178227 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-e129da0d-51fe-4430-ad00-6fafc1b130ac admin admin] The token cache is being invalidate because project 51c380340d5446679239a1103a7ace03 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80119) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 10:33:14.178520 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-e129da0d-51fe-4430-ad00-6fafc1b130ac admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:33:14.179238 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-e129da0d-51fe-4430-ad00-6fafc1b130ac admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:33:14.180922 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1096/2184] 10.222.0.49 () {58 vars in 1232 bytes} [Wed Jun 15 10:33:14 2022] DELETE /identity/v3/projects/51c380340d5446679239a1103a7ace03 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:33:14.185997 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edf4dbf-62dc-4b00-adfa-14c3a2ebf2ac None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:33:14.186328 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edf4dbf-62dc-4b00-adfa-14c3a2ebf2ac None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:33:14.186605 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edf4dbf-62dc-4b00-adfa-14c3a2ebf2ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:33:14.204554 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-4edf4dbf-62dc-4b00-adfa-14c3a2ebf2ac None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:33:14.205760 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-4edf4dbf-62dc-4b00-adfa-14c3a2ebf2ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:33:14.215314 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:33:14.270777 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1089/2185] 10.222.0.49 () {58 vars in 957 bytes} [Wed Jun 15 10:33:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 10:33:14.278894 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfb847e-26d7-4087-9baa-61ba3b4c579f None None] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:33:14.279222 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfb847e-26d7-4087-9baa-61ba3b4c579f None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:33:14.279504 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfb847e-26d7-4087-9baa-61ba3b4c579f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:33:14.299789 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.auth.core [None req-acfb847e-26d7-4087-9baa-61ba3b4c579f None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:33:14.299789 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-acfb847e-26d7-4087-9baa-61ba3b4c579f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:33:14.307046 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1097/2186] 10.222.0.49 () {58 vars in 957 bytes} [Wed Jun 15 10:33:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:33:14.311133 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4edf4dbf-62dc-4b00-adfa-14c3a2ebf2ac None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:33:14.321486 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4edf4dbf-62dc-4b00-adfa-14c3a2ebf2ac None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:33:14.325970 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653e4f77-4d7f-4fb1-9ec9-a61db313b69e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:33:14.326617 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653e4f77-4d7f-4fb1-9ec9-a61db313b69e admin admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:33:14.326906 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653e4f77-4d7f-4fb1-9ec9-a61db313b69e admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:33:14.327212 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653e4f77-4d7f-4fb1-9ec9-a61db313b69e admin admin] PATH_INFO: `/v3/domains` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:33:14.328015 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-653e4f77-4d7f-4fb1-9ec9-a61db313b69e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:33:14.328321 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-653e4f77-4d7f-4fb1-9ec9-a61db313b69e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:33:14.329905 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-653e4f77-4d7f-4fb1-9ec9-a61db313b69e admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:33:14.346040 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1090/2187] 10.222.0.49 () {58 vars in 1152 bytes} [Wed Jun 15 10:33:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:33:14.350193 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acfb847e-26d7-4087-9baa-61ba3b4c579f None None] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:33:14.359429 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acfb847e-26d7-4087-9baa-61ba3b4c579f None None] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:33:14.364024 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66f02e0-6191-4193-a68e-525d7a93581e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:33:14.364672 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66f02e0-6191-4193-a68e-525d7a93581e admin admin] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:33:14.364952 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66f02e0-6191-4193-a68e-525d7a93581e admin admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:33:14.365227 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66f02e0-6191-4193-a68e-525d7a93581e admin admin] PATH_INFO: `/v3/projects` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:33:14.365906 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a66f02e0-6191-4193-a68e-525d7a93581e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:33:14.367901 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a66f02e0-6191-4193-a68e-525d7a93581e admin admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:33:14.393748 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1098/2188] 10.222.0.49 () {60 vars in 1152 bytes} [Wed Jun 15 10:33:14 2022] POST /identity/v3/projects => generated 383 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 10:33:14.398406 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653e4f77-4d7f-4fb1-9ec9-a61db313b69e admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:33:14.408654 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653e4f77-4d7f-4fb1-9ec9-a61db313b69e admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:33:14.465508 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd9b02f-5996-4827-af9c-7257461072d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:33:14.466174 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd9b02f-5996-4827-af9c-7257461072d3 admin admin] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:33:14.466455 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd9b02f-5996-4827-af9c-7257461072d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:33:14.466729 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd9b02f-5996-4827-af9c-7257461072d3 admin admin] PATH_INFO: `/v3/users` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:33:14.467465 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd9b02f-5996-4827-af9c-7257461072d3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:33:14.469389 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd9b02f-5996-4827-af9c-7257461072d3 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:33:14.488434 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1091/2189] 10.222.0.49 () {60 vars in 1143 bytes} [Wed Jun 15 10:33:14 2022] POST /identity/v3/users => generated 292 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 10:33:14.493072 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66f02e0-6191-4193-a68e-525d7a93581e admin admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:33:14.503650 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66f02e0-6191-4193-a68e-525d7a93581e admin admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:33:14.508844 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d0f82b-c7dc-49ad-9578-24b5a063fb9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:33:14.509528 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d0f82b-c7dc-49ad-9578-24b5a063fb9a admin admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:33:14.509813 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d0f82b-c7dc-49ad-9578-24b5a063fb9a admin admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:33:14.510088 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d0f82b-c7dc-49ad-9578-24b5a063fb9a admin admin] PATH_INFO: `/v3/roles` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:33:14.510813 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d0f82b-c7dc-49ad-9578-24b5a063fb9a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:33:14.512463 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d0f82b-c7dc-49ad-9578-24b5a063fb9a admin admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:33:14.522734 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1099/2190] 10.222.0.49 () {58 vars in 1121 bytes} [Wed Jun 15 10:33:14 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 15 10:33:14.527906 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd9b02f-5996-4827-af9c-7257461072d3 admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:33:14.537990 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd9b02f-5996-4827-af9c-7257461072d3 admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:33:14.542766 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9583f7-90b9-496f-b139-1bc9ced2607b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:33:14.543391 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9583f7-90b9-496f-b139-1bc9ced2607b admin admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:33:14.543676 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9583f7-90b9-496f-b139-1bc9ced2607b admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:33:14.543950 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9583f7-90b9-496f-b139-1bc9ced2607b admin admin] PATH_INFO: `/v3/roles` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:33:14.544672 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df9583f7-90b9-496f-b139-1bc9ced2607b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:33:14.546243 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df9583f7-90b9-496f-b139-1bc9ced2607b admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:33:14.556511 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1092/2191] 10.222.0.49 () {58 vars in 1121 bytes} [Wed Jun 15 10:33:14 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 10:33:14.560460 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d0f82b-c7dc-49ad-9578-24b5a063fb9a admin admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:33:14.570842 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d0f82b-c7dc-49ad-9578-24b5a063fb9a admin admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:33:14.575473 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b002e478-fe33-4b6a-8ed9-3d79f9f580eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:33:14.576356 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b002e478-fe33-4b6a-8ed9-3d79f9f580eb admin admin] REQUEST_METHOD: `PUT` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:33:14.576640 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b002e478-fe33-4b6a-8ed9-3d79f9f580eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:33:14.576920 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b002e478-fe33-4b6a-8ed9-3d79f9f580eb admin admin] PATH_INFO: `/v3/projects/440f69d2725e435ab11b2fc658e1e745/users/572ac94e242f414aab152286fa356335/roles/6876e57cc51c448097ab109d35b2a3da` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:33:14.592404 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b002e478-fe33-4b6a-8ed9-3d79f9f580eb admin admin] RBAC: Authorizing `identity:create_grant(project_id=440f69d2725e435ab11b2fc658e1e745, user_id=572ac94e242f414aab152286fa356335, role_id=6876e57cc51c448097ab109d35b2a3da)` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:33:14.595217 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b002e478-fe33-4b6a-8ed9-3d79f9f580eb admin admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:33:14.607875 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1100/2192] 10.222.0.49 () {60 vars in 1482 bytes} [Wed Jun 15 10:33:14 2022] PUT /identity/v3/projects/440f69d2725e435ab11b2fc658e1e745/users/572ac94e242f414aab152286fa356335/roles/6876e57cc51c448097ab109d35b2a3da => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:33:14.612906 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ad1ee7-3fca-4ea6-a8e9-ea2fbdc16cfa None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:33:14.613214 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ad1ee7-3fca-4ea6-a8e9-ea2fbdc16cfa None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:33:14.613505 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ad1ee7-3fca-4ea6-a8e9-ea2fbdc16cfa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:33:14.631475 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-c8ad1ee7-3fca-4ea6-a8e9-ea2fbdc16cfa None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:33:14.633326 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-c8ad1ee7-3fca-4ea6-a8e9-ea2fbdc16cfa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:33:14.690263 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1093/2193] 10.222.0.49 () {58 vars in 957 bytes} [Wed Jun 15 10:33:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:33:14.700079 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b002e478-fe33-4b6a-8ed9-3d79f9f580eb admin admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:33:14.701757 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-b002e478-fe33-4b6a-8ed9-3d79f9f580eb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:33:14.756855 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b002e478-fe33-4b6a-8ed9-3d79f9f580eb admin admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:33:14.761194 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0488cf7-5345-40bb-a9e9-1ecda6ae9e70 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0cb44a17d205431793e9a8791e477a32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '559b3e713f5844b6ae37151d56dd419f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:33:14.761829 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0488cf7-5345-40bb-a9e9-1ecda6ae9e70 service neutron] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:33:14.762112 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0488cf7-5345-40bb-a9e9-1ecda6ae9e70 service neutron] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:33:14.762511 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0488cf7-5345-40bb-a9e9-1ecda6ae9e70 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:33:14.771359 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0488cf7-5345-40bb-a9e9-1ecda6ae9e70 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:33:14.773528 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0488cf7-5345-40bb-a9e9-1ecda6ae9e70 service neutron] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:33:14.783980 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1101/2194] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Jun 15 10:33:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:33:19.170396 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b28b30-c0b6-41e0-b6e2-4c077981f87e None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:33:19.170861 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b28b30-c0b6-41e0-b6e2-4c077981f87e None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:33:19.170986 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b28b30-c0b6-41e0-b6e2-4c077981f87e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:33:19.211734 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-a8b28b30-c0b6-41e0-b6e2-4c077981f87e None None] MFA Rules not processed for user `572ac94e242f414aab152286fa356335`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:33:19.212911 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-a8b28b30-c0b6-41e0-b6e2-4c077981f87e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:33:19.222365 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:33:19.291321 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1094/2195] 10.222.0.49 () {58 vars in 957 bytes} [Wed Jun 15 10:33:19 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:33:19.295954 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0488cf7-5345-40bb-a9e9-1ecda6ae9e70 service neutron] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:33:19.306627 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0488cf7-5345-40bb-a9e9-1ecda6ae9e70 service neutron] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:33:19.311463 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cceb13f7-b551-49b6-b421-204159279858 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:33:19.312886 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cceb13f7-b551-49b6-b421-204159279858 admin admin] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:33:19.313177 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cceb13f7-b551-49b6-b421-204159279858 admin admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:33:19.313457 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cceb13f7-b551-49b6-b421-204159279858 admin admin] PATH_INFO: `/v3/projects` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:33:19.314434 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cceb13f7-b551-49b6-b421-204159279858 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:33:19.316352 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cceb13f7-b551-49b6-b421-204159279858 admin admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:33:19.346370 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1102/2196] 10.222.0.49 () {60 vars in 1152 bytes} [Wed Jun 15 10:33:19 2022] POST /identity/v3/projects => generated 385 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 10:33:19.350308 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b28b30-c0b6-41e0-b6e2-4c077981f87e None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:33:19.361559 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b28b30-c0b6-41e0-b6e2-4c077981f87e None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:33:19.418553 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea468ada-8a98-46f6-b8c4-d67d6213acb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:33:19.419257 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea468ada-8a98-46f6-b8c4-d67d6213acb8 admin admin] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:33:19.419549 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea468ada-8a98-46f6-b8c4-d67d6213acb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:33:19.419831 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea468ada-8a98-46f6-b8c4-d67d6213acb8 admin admin] PATH_INFO: `/v3/users` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:33:19.420517 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea468ada-8a98-46f6-b8c4-d67d6213acb8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:33:19.422443 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea468ada-8a98-46f6-b8c4-d67d6213acb8 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:33:19.444962 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1095/2197] 10.222.0.49 () {60 vars in 1143 bytes} [Wed Jun 15 10:33:19 2022] POST /identity/v3/users => generated 299 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 10:33:19.446000 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cceb13f7-b551-49b6-b421-204159279858 admin admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:33:19.457185 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cceb13f7-b551-49b6-b421-204159279858 admin admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:33:19.462214 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0fe55a-10e6-44c2-ba11-0c7d626ab7ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:33:19.462852 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0fe55a-10e6-44c2-ba11-0c7d626ab7ea admin admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:33:19.463170 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0fe55a-10e6-44c2-ba11-0c7d626ab7ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:33:19.463515 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0fe55a-10e6-44c2-ba11-0c7d626ab7ea admin admin] PATH_INFO: `/v3/roles` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:33:19.464247 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca0fe55a-10e6-44c2-ba11-0c7d626ab7ea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:33:19.465994 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca0fe55a-10e6-44c2-ba11-0c7d626ab7ea admin admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:33:19.468653 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:33:19.483438 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1103/2198] 10.222.0.49 () {58 vars in 1121 bytes} [Wed Jun 15 10:33:19 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 15 10:33:19.485428 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea468ada-8a98-46f6-b8c4-d67d6213acb8 admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:33:19.496273 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea468ada-8a98-46f6-b8c4-d67d6213acb8 admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:33:19.501448 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7cbfd5f-1f1b-4430-870d-695966177a55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:33:19.502367 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7cbfd5f-1f1b-4430-870d-695966177a55 admin admin] REQUEST_METHOD: `PUT` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:33:19.502649 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7cbfd5f-1f1b-4430-870d-695966177a55 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:33:19.502925 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7cbfd5f-1f1b-4430-870d-695966177a55 admin admin] PATH_INFO: `/v3/projects/ed7082a13129477c8ff6630ced935884/users/412744e697f94346aeeabe2a1422aa53/roles/8051174e7a7947d5b991ebd793c2e543` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:33:19.518100 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7cbfd5f-1f1b-4430-870d-695966177a55 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ed7082a13129477c8ff6630ced935884, user_id=412744e697f94346aeeabe2a1422aa53, role_id=8051174e7a7947d5b991ebd793c2e543)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:33:19.520819 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7cbfd5f-1f1b-4430-870d-695966177a55 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:33:19.535336 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1096/2199] 10.222.0.49 () {60 vars in 1482 bytes} [Wed Jun 15 10:33:19 2022] PUT /identity/v3/projects/ed7082a13129477c8ff6630ced935884/users/412744e697f94346aeeabe2a1422aa53/roles/8051174e7a7947d5b991ebd793c2e543 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:33:19.539830 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0fe55a-10e6-44c2-ba11-0c7d626ab7ea admin admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:33:19.550572 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0fe55a-10e6-44c2-ba11-0c7d626ab7ea admin admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:33:19.613615 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4cc1ed-607c-4e74-8d50-1a929573cd32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:33:19.614537 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4cc1ed-607c-4e74-8d50-1a929573cd32 admin admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:33:19.614808 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4cc1ed-607c-4e74-8d50-1a929573cd32 admin admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:33:19.615161 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4cc1ed-607c-4e74-8d50-1a929573cd32 admin admin] PATH_INFO: `/v3/roles` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:33:19.615991 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b4cc1ed-607c-4e74-8d50-1a929573cd32 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:33:19.618204 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b4cc1ed-607c-4e74-8d50-1a929573cd32 admin admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:33:19.631590 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1104/2200] 10.222.0.49 () {58 vars in 1121 bytes} [Wed Jun 15 10:33:19 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 10:33:19.632904 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7cbfd5f-1f1b-4430-870d-695966177a55 admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:33:19.643318 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7cbfd5f-1f1b-4430-870d-695966177a55 admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:33:19.648223 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0100d59-e8bc-4991-8057-a5c0640a55c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:33:19.649140 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0100d59-e8bc-4991-8057-a5c0640a55c8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:33:19.649431 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0100d59-e8bc-4991-8057-a5c0640a55c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:33:19.649815 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0100d59-e8bc-4991-8057-a5c0640a55c8 admin admin] PATH_INFO: `/v3/projects/ed7082a13129477c8ff6630ced935884/users/412744e697f94346aeeabe2a1422aa53/roles/8051174e7a7947d5b991ebd793c2e543` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:33:19.651814 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0100d59-e8bc-4991-8057-a5c0640a55c8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ed7082a13129477c8ff6630ced935884, user_id=412744e697f94346aeeabe2a1422aa53, role_id=8051174e7a7947d5b991ebd793c2e543)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:33:19.654435 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0100d59-e8bc-4991-8057-a5c0640a55c8 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:33:19.666981 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1097/2201] 10.222.0.49 () {60 vars in 1482 bytes} [Wed Jun 15 10:33:19 2022] PUT /identity/v3/projects/ed7082a13129477c8ff6630ced935884/users/412744e697f94346aeeabe2a1422aa53/roles/8051174e7a7947d5b991ebd793c2e543 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:33:24.454645 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4cc1ed-607c-4e74-8d50-1a929573cd32 admin admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:33:24.456601 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-0b4cc1ed-607c-4e74-8d50-1a929573cd32 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:33:24.480749 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4cc1ed-607c-4e74-8d50-1a929573cd32 admin admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:33:24.554438 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca38fcd-a98f-4717-b581-5fbc5c9d8b7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:33:24.555374 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca38fcd-a98f-4717-b581-5fbc5c9d8b7e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:33:24.555650 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca38fcd-a98f-4717-b581-5fbc5c9d8b7e admin admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:33:24.555931 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca38fcd-a98f-4717-b581-5fbc5c9d8b7e admin admin] PATH_INFO: `/v3/users/cb44dcb7cba44464ad9f731b41269802` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:33:24.557255 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca38fcd-a98f-4717-b581-5fbc5c9d8b7e admin admin] RBAC: Authorizing `identity:delete_user(user_id=cb44dcb7cba44464ad9f731b41269802)` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:33:24.559301 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca38fcd-a98f-4717-b581-5fbc5c9d8b7e admin admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:33:24.649220 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-dca38fcd-a98f-4717-b581-5fbc5c9d8b7e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cb44dcb7cba44464ad9f731b41269802'} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:33:24.661089 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-dca38fcd-a98f-4717-b581-5fbc5c9d8b7e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cb44dcb7cba44464ad9f731b41269802'} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:33:24.662554 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-dca38fcd-a98f-4717-b581-5fbc5c9d8b7e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cb44dcb7cba44464ad9f731b41269802'} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:33:24.664921 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-dca38fcd-a98f-4717-b581-5fbc5c9d8b7e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cb44dcb7cba44464ad9f731b41269802'} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:33:24.690368 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-dca38fcd-a98f-4717-b581-5fbc5c9d8b7e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cb44dcb7cba44464ad9f731b41269802'} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:33:24.774145 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1105/2202] 10.222.0.49 () {58 vars in 1223 bytes} [Wed Jun 15 10:33:24 2022] DELETE /identity/v3/users/cb44dcb7cba44464ad9f731b41269802 => generated 0 bytes in 318 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:33:24.776913 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bcfce6-3418-4fec-a6bb-87e51ba570d5 None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:33:24.777324 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bcfce6-3418-4fec-a6bb-87e51ba570d5 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:33:24.777324 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bcfce6-3418-4fec-a6bb-87e51ba570d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:33:24.817499 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-94bcfce6-3418-4fec-a6bb-87e51ba570d5 None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:33:24.820879 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-94bcfce6-3418-4fec-a6bb-87e51ba570d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:33:24.939332 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1098/2203] 10.222.0.49 () {58 vars in 957 bytes} [Wed Jun 15 10:33:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:33:24.969703 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca38fcd-a98f-4717-b581-5fbc5c9d8b7e admin admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:33:24.971220 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-dca38fcd-a98f-4717-b581-5fbc5c9d8b7e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:33:24.996249 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:33:25.114354 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca38fcd-a98f-4717-b581-5fbc5c9d8b7e admin admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:33:25.124030 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13937d1d-5728-4928-ac19-4c21b010b29b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0cb44a17d205431793e9a8791e477a32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '559b3e713f5844b6ae37151d56dd419f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:33:25.124723 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13937d1d-5728-4928-ac19-4c21b010b29b service neutron] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:33:25.125037 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13937d1d-5728-4928-ac19-4c21b010b29b service neutron] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:33:25.125324 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13937d1d-5728-4928-ac19-4c21b010b29b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:33:25.152945 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13937d1d-5728-4928-ac19-4c21b010b29b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:33:25.154596 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13937d1d-5728-4928-ac19-4c21b010b29b service neutron] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:33:25.200904 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1106/2204] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Jun 15 10:33:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 232 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:33:25.312009 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b180396-227a-4eee-bf29-f89b56716bf1 None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:33:25.312303 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b180396-227a-4eee-bf29-f89b56716bf1 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:33:25.312584 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b180396-227a-4eee-bf29-f89b56716bf1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:33:25.319256 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:33:25.362457 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-9b180396-227a-4eee-bf29-f89b56716bf1 None None] MFA Rules not processed for user `412744e697f94346aeeabe2a1422aa53`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:33:25.363882 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-9b180396-227a-4eee-bf29-f89b56716bf1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:33:25.471608 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1099/2205] 10.222.0.49 () {58 vars in 957 bytes} [Wed Jun 15 10:33:25 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:33:25.493581 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13937d1d-5728-4928-ac19-4c21b010b29b service neutron] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:33:25.495141 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-13937d1d-5728-4928-ac19-4c21b010b29b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:33:25.585011 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13937d1d-5728-4928-ac19-4c21b010b29b service neutron] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:33:25.591202 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c221f66-8101-486b-97b3-b323a5fb9de5 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91f2fe3be62e49548eae208e57bf2a93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '559b3e713f5844b6ae37151d56dd419f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:33:25.592310 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c221f66-8101-486b-97b3-b323a5fb9de5 service glance] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:33:25.592881 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c221f66-8101-486b-97b3-b323a5fb9de5 service glance] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:33:25.593186 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c221f66-8101-486b-97b3-b323a5fb9de5 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:33:25.595494 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-6c221f66-8101-486b-97b3-b323a5fb9de5 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:33:25.697723 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c221f66-8101-486b-97b3-b323a5fb9de5 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:33:25.702022 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c221f66-8101-486b-97b3-b323a5fb9de5 service glance] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:33:25.733324 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1107/2206] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Jun 15 10:33:25 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 238 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:33:25.846394 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b180396-227a-4eee-bf29-f89b56716bf1 None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:33:25.847941 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-9b180396-227a-4eee-bf29-f89b56716bf1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:33:25.874101 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b180396-227a-4eee-bf29-f89b56716bf1 None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:33:25.892532 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20114d44-1a56-44d4-b7b9-0fcc35350382 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:33:25.893453 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20114d44-1a56-44d4-b7b9-0fcc35350382 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:33:25.893742 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20114d44-1a56-44d4-b7b9-0fcc35350382 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:33:25.894323 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20114d44-1a56-44d4-b7b9-0fcc35350382 admin admin] PATH_INFO: `/v3/projects/c83fd32c69114456a9a01cceaadb6686` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:33:25.896135 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20114d44-1a56-44d4-b7b9-0fcc35350382 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c83fd32c69114456a9a01cceaadb6686)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:33:25.899022 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20114d44-1a56-44d4-b7b9-0fcc35350382 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:33:25.975408 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824b5d0c-c197-43ce-8a64-a0dc5bfa2f88 None None] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:33:25.976456 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824b5d0c-c197-43ce-8a64-a0dc5bfa2f88 None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:33:25.976878 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824b5d0c-c197-43ce-8a64-a0dc5bfa2f88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:33:26.011917 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.auth.core [None req-824b5d0c-c197-43ce-8a64-a0dc5bfa2f88 None None] MFA Rules not processed for user `103e10244eca444b86b9c51ff314d445`. Rule list: `[]` (Enabled: `True`). {{(pid=80119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:33:26.013441 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-824b5d0c-c197-43ce-8a64-a0dc5bfa2f88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:33:26.041945 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-20114d44-1a56-44d4-b7b9-0fcc35350382 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c83fd32c69114456a9a01cceaadb6686', 'request_id': 'req-20114d44-1a56-44d4-b7b9-0fcc35350382'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:33:26.054399 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-20114d44-1a56-44d4-b7b9-0fcc35350382 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c83fd32c69114456a9a01cceaadb6686', 'request_id': 'req-20114d44-1a56-44d4-b7b9-0fcc35350382'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:33:26.065241 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-20114d44-1a56-44d4-b7b9-0fcc35350382 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c83fd32c69114456a9a01cceaadb6686', 'request_id': 'req-20114d44-1a56-44d4-b7b9-0fcc35350382'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:33:26.081827 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-20114d44-1a56-44d4-b7b9-0fcc35350382 admin admin] The token cache is being invalidate because project c83fd32c69114456a9a01cceaadb6686 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80120) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 10:33:26.082518 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-20114d44-1a56-44d4-b7b9-0fcc35350382 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:33:26.082887 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-20114d44-1a56-44d4-b7b9-0fcc35350382 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:33:26.091473 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1100/2207] 10.222.0.49 () {58 vars in 1232 bytes} [Wed Jun 15 10:33:25 2022] DELETE /identity/v3/projects/c83fd32c69114456a9a01cceaadb6686 => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:33:26.109325 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1108/2208] 10.222.0.49 () {58 vars in 1009 bytes} [Wed Jun 15 10:33:25 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:33:26.160766 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20114d44-1a56-44d4-b7b9-0fcc35350382 admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:33:26.162227 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-20114d44-1a56-44d4-b7b9-0fcc35350382 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:33:26.248991 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20114d44-1a56-44d4-b7b9-0fcc35350382 admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:33:26.254232 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd04019b-d73d-485a-8ad0-27e751e3636f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a69f99210444f9696e8b22615fb4a41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '559b3e713f5844b6ae37151d56dd419f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:33:26.254908 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd04019b-d73d-485a-8ad0-27e751e3636f service swift] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:33:26.255235 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd04019b-d73d-485a-8ad0-27e751e3636f service swift] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:33:26.255515 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd04019b-d73d-485a-8ad0-27e751e3636f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:33:26.265199 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd04019b-d73d-485a-8ad0-27e751e3636f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:33:26.266812 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd04019b-d73d-485a-8ad0-27e751e3636f service swift] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:33:26.271890 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:33:26.276981 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1101/2209] 10.222.0.49 () {60 vars in 1384 bytes} [Wed Jun 15 10:33:26 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 15 10:33:33.277714 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f463ecd-f9aa-4991-82e9-0ba6f7c0f753 None None] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:33:33.278327 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f463ecd-f9aa-4991-82e9-0ba6f7c0f753 None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:33:33.278327 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f463ecd-f9aa-4991-82e9-0ba6f7c0f753 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:33:33.301399 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.auth.core [None req-0f463ecd-f9aa-4991-82e9-0ba6f7c0f753 None None] MFA Rules not processed for user `14b720a2d0854d218272390205887100`. Rule list: `[]` (Enabled: `True`). {{(pid=80119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:33:33.302675 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-0f463ecd-f9aa-4991-82e9-0ba6f7c0f753 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:33:33.313123 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=100 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:33:33.367126 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1109/2210] 10.222.0.49 () {58 vars in 957 bytes} [Wed Jun 15 10:33:33 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 10:33:34.629043 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd04019b-d73d-485a-8ad0-27e751e3636f service swift] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:33:34.631938 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-fd04019b-d73d-485a-8ad0-27e751e3636f service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:33:34.706858 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd04019b-d73d-485a-8ad0-27e751e3636f service swift] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:33:34.712460 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9f3bdd-3fc7-45f8-a8eb-f257e92742a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:33:34.713540 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9f3bdd-3fc7-45f8-a8eb-f257e92742a3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:33:34.713912 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9f3bdd-3fc7-45f8-a8eb-f257e92742a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:33:34.714200 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9f3bdd-3fc7-45f8-a8eb-f257e92742a3 admin admin] PATH_INFO: `/v3/users/5c39c8958eb64553bfad994b4eb21db8` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:33:34.715835 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9f3bdd-3fc7-45f8-a8eb-f257e92742a3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5c39c8958eb64553bfad994b4eb21db8)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:33:34.719077 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9f3bdd-3fc7-45f8-a8eb-f257e92742a3 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:33:34.822490 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-4e9f3bdd-3fc7-45f8-a8eb-f257e92742a3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5c39c8958eb64553bfad994b4eb21db8'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:33:34.830878 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-4e9f3bdd-3fc7-45f8-a8eb-f257e92742a3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5c39c8958eb64553bfad994b4eb21db8'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:33:34.832124 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-4e9f3bdd-3fc7-45f8-a8eb-f257e92742a3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5c39c8958eb64553bfad994b4eb21db8'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:33:34.832940 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-4e9f3bdd-3fc7-45f8-a8eb-f257e92742a3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5c39c8958eb64553bfad994b4eb21db8'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:33:34.847178 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-4e9f3bdd-3fc7-45f8-a8eb-f257e92742a3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5c39c8958eb64553bfad994b4eb21db8'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:33:34.872541 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1102/2211] 10.222.0.49 () {58 vars in 1223 bytes} [Wed Jun 15 10:33:34 2022] DELETE /identity/v3/users/5c39c8958eb64553bfad994b4eb21db8 => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:33:34.876757 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2e7a42-b389-4546-b532-a25e9ffb74ab None None] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:33:34.877260 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2e7a42-b389-4546-b532-a25e9ffb74ab None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:33:34.877356 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2e7a42-b389-4546-b532-a25e9ffb74ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:33:34.901645 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.auth.core [None req-7a2e7a42-b389-4546-b532-a25e9ffb74ab None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:33:34.903725 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-7a2e7a42-b389-4546-b532-a25e9ffb74ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:33:35.041205 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1110/2212] 10.222.0.49 () {58 vars in 957 bytes} [Wed Jun 15 10:33:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 10:33:35.041809 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9f3bdd-3fc7-45f8-a8eb-f257e92742a3 admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:33:35.043407 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-4e9f3bdd-3fc7-45f8-a8eb-f257e92742a3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:33:35.142795 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9f3bdd-3fc7-45f8-a8eb-f257e92742a3 admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:33:35.163314 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468eca4d-484b-43b7-90f5-b3004ae24a4e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0cb44a17d205431793e9a8791e477a32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '559b3e713f5844b6ae37151d56dd419f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:33:35.163996 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468eca4d-484b-43b7-90f5-b3004ae24a4e service neutron] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:33:35.164284 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468eca4d-484b-43b7-90f5-b3004ae24a4e service neutron] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:33:35.164568 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468eca4d-484b-43b7-90f5-b3004ae24a4e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:33:35.191483 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-468eca4d-484b-43b7-90f5-b3004ae24a4e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:33:35.193210 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-468eca4d-484b-43b7-90f5-b3004ae24a4e service neutron] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:33:35.206145 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1103/2213] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Jun 15 10:33:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:33:35.827648 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2e7a42-b389-4546-b532-a25e9ffb74ab None None] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:33:35.830230 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-7a2e7a42-b389-4546-b532-a25e9ffb74ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:33:35.861544 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2e7a42-b389-4546-b532-a25e9ffb74ab None None] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:33:35.883862 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddcf978b-bd5a-44c0-b265-c61e803ae15b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:33:35.884342 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcf978b-bd5a-44c0-b265-c61e803ae15b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:33:35.884342 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcf978b-bd5a-44c0-b265-c61e803ae15b admin admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:33:35.884601 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcf978b-bd5a-44c0-b265-c61e803ae15b admin admin] PATH_INFO: `/v3/projects/f655ca9cf258481d8fe8b85af5c80392` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:33:35.885965 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddcf978b-bd5a-44c0-b265-c61e803ae15b admin admin] RBAC: Authorizing `identity:delete_project(project_id=f655ca9cf258481d8fe8b85af5c80392)` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:33:35.888040 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddcf978b-bd5a-44c0-b265-c61e803ae15b admin admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:33:35.955638 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-ddcf978b-bd5a-44c0-b265-c61e803ae15b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f655ca9cf258481d8fe8b85af5c80392', 'request_id': 'req-ddcf978b-bd5a-44c0-b265-c61e803ae15b'} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:33:35.965419 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-ddcf978b-bd5a-44c0-b265-c61e803ae15b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f655ca9cf258481d8fe8b85af5c80392', 'request_id': 'req-ddcf978b-bd5a-44c0-b265-c61e803ae15b'} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:33:35.972586 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-ddcf978b-bd5a-44c0-b265-c61e803ae15b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f655ca9cf258481d8fe8b85af5c80392', 'request_id': 'req-ddcf978b-bd5a-44c0-b265-c61e803ae15b'} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:33:35.986767 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-ddcf978b-bd5a-44c0-b265-c61e803ae15b admin admin] The token cache is being invalidate because project f655ca9cf258481d8fe8b85af5c80392 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80119) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 10:33:35.987128 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-ddcf978b-bd5a-44c0-b265-c61e803ae15b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:33:35.987787 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-ddcf978b-bd5a-44c0-b265-c61e803ae15b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:33:35.989678 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1111/2214] 10.222.0.49 () {58 vars in 1232 bytes} [Wed Jun 15 10:33:35 2022] DELETE /identity/v3/projects/f655ca9cf258481d8fe8b85af5c80392 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:33:35.995107 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3e7705-4b40-4135-b84f-114c9ae56e68 None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:33:35.995581 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3e7705-4b40-4135-b84f-114c9ae56e68 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:33:35.995820 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3e7705-4b40-4135-b84f-114c9ae56e68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:33:36.016869 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-7f3e7705-4b40-4135-b84f-114c9ae56e68 None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:33:36.018187 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-7f3e7705-4b40-4135-b84f-114c9ae56e68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:33:36.038944 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:33:36.097976 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1104/2215] 10.222.0.49 () {58 vars in 957 bytes} [Wed Jun 15 10:33:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:33:36.100028 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71353fba-1dc1-4c86-8d48-58b9ac45df6c None None] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:33:36.100406 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71353fba-1dc1-4c86-8d48-58b9ac45df6c None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:33:36.100667 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71353fba-1dc1-4c86-8d48-58b9ac45df6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:33:36.132224 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.auth.core [None req-71353fba-1dc1-4c86-8d48-58b9ac45df6c None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:33:36.133793 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-71353fba-1dc1-4c86-8d48-58b9ac45df6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:33:36.162104 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1112/2216] 10.222.0.49 () {58 vars in 957 bytes} [Wed Jun 15 10:33:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 63 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:33:36.166397 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3e7705-4b40-4135-b84f-114c9ae56e68 None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:33:36.205582 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3e7705-4b40-4135-b84f-114c9ae56e68 None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:33:36.212132 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9310e78d-239e-4547-918a-1bad0448d1d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:33:36.212835 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9310e78d-239e-4547-918a-1bad0448d1d6 admin admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:33:36.213705 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9310e78d-239e-4547-918a-1bad0448d1d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:33:36.213705 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9310e78d-239e-4547-918a-1bad0448d1d6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:33:36.214137 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9310e78d-239e-4547-918a-1bad0448d1d6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:33:36.214450 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9310e78d-239e-4547-918a-1bad0448d1d6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:33:36.216340 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9310e78d-239e-4547-918a-1bad0448d1d6 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:33:36.241913 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1105/2217] 10.222.0.49 () {58 vars in 1152 bytes} [Wed Jun 15 10:33:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:33:36.242149 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71353fba-1dc1-4c86-8d48-58b9ac45df6c None None] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:33:36.248407 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71353fba-1dc1-4c86-8d48-58b9ac45df6c None None] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:33:36.253662 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb61871-7c77-445b-b264-555c7bf5d626 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:33:36.254484 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb61871-7c77-445b-b264-555c7bf5d626 admin admin] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:33:36.254775 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb61871-7c77-445b-b264-555c7bf5d626 admin admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:33:36.255056 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb61871-7c77-445b-b264-555c7bf5d626 admin admin] PATH_INFO: `/v3/projects` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:33:36.256029 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb61871-7c77-445b-b264-555c7bf5d626 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:33:36.258520 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb61871-7c77-445b-b264-555c7bf5d626 admin admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:33:36.332889 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1113/2218] 10.222.0.49 () {60 vars in 1152 bytes} [Wed Jun 15 10:33:36 2022] POST /identity/v3/projects => generated 371 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 10:33:36.334782 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9310e78d-239e-4547-918a-1bad0448d1d6 admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:33:36.346506 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9310e78d-239e-4547-918a-1bad0448d1d6 admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:33:36.451430 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375004eb-2382-4e9a-a3d7-a78c8e5fe037 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:33:36.452116 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375004eb-2382-4e9a-a3d7-a78c8e5fe037 admin admin] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:33:36.452404 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375004eb-2382-4e9a-a3d7-a78c8e5fe037 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:33:36.452683 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375004eb-2382-4e9a-a3d7-a78c8e5fe037 admin admin] PATH_INFO: `/v3/users` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:33:36.453375 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375004eb-2382-4e9a-a3d7-a78c8e5fe037 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:33:36.455365 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375004eb-2382-4e9a-a3d7-a78c8e5fe037 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:33:36.487586 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1106/2219] 10.222.0.49 () {60 vars in 1143 bytes} [Wed Jun 15 10:33:36 2022] POST /identity/v3/users => generated 286 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 10:33:36.489123 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb61871-7c77-445b-b264-555c7bf5d626 admin admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:33:36.501064 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb61871-7c77-445b-b264-555c7bf5d626 admin admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:33:36.506169 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76136834-005e-4364-8f25-0cd76e93a59f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:33:36.507966 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76136834-005e-4364-8f25-0cd76e93a59f admin admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:33:36.508792 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76136834-005e-4364-8f25-0cd76e93a59f admin admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:33:36.509285 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76136834-005e-4364-8f25-0cd76e93a59f admin admin] PATH_INFO: `/v3/roles` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:33:36.513184 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76136834-005e-4364-8f25-0cd76e93a59f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:33:36.514920 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76136834-005e-4364-8f25-0cd76e93a59f admin admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:33:36.543839 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375004eb-2382-4e9a-a3d7-a78c8e5fe037 admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:33:36.553724 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1114/2220] 10.222.0.49 () {58 vars in 1121 bytes} [Wed Jun 15 10:33:36 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 10:33:36.558887 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375004eb-2382-4e9a-a3d7-a78c8e5fe037 admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:33:36.564580 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967973a7-6203-4ad9-af12-879919591511 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:33:36.567373 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967973a7-6203-4ad9-af12-879919591511 admin admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:33:36.567741 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967973a7-6203-4ad9-af12-879919591511 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:33:36.568113 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967973a7-6203-4ad9-af12-879919591511 admin admin] PATH_INFO: `/v3/roles` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:33:36.569710 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-967973a7-6203-4ad9-af12-879919591511 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:33:36.571622 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-967973a7-6203-4ad9-af12-879919591511 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:33:36.589295 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76136834-005e-4364-8f25-0cd76e93a59f admin admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:33:36.596310 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1107/2221] 10.222.0.49 () {58 vars in 1121 bytes} [Wed Jun 15 10:33:36 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 10:33:36.596746 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:33:36.604349 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76136834-005e-4364-8f25-0cd76e93a59f admin admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:33:36.613764 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259639fe-c788-45a4-8410-1de4722cb5a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:33:36.614993 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259639fe-c788-45a4-8410-1de4722cb5a1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:33:36.615463 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259639fe-c788-45a4-8410-1de4722cb5a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:33:36.616379 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259639fe-c788-45a4-8410-1de4722cb5a1 admin admin] PATH_INFO: `/v3/projects/2e616955fbdf473ea39c76678b402087/users/37ce86ae50d84108b204e3c31b28f7d7/roles/6876e57cc51c448097ab109d35b2a3da` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:33:36.639822 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-259639fe-c788-45a4-8410-1de4722cb5a1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2e616955fbdf473ea39c76678b402087, user_id=37ce86ae50d84108b204e3c31b28f7d7, role_id=6876e57cc51c448097ab109d35b2a3da)` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:33:36.643537 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-259639fe-c788-45a4-8410-1de4722cb5a1 admin admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:33:36.672646 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1115/2222] 10.222.0.49 () {60 vars in 1482 bytes} [Wed Jun 15 10:33:36 2022] PUT /identity/v3/projects/2e616955fbdf473ea39c76678b402087/users/37ce86ae50d84108b204e3c31b28f7d7/roles/6876e57cc51c448097ab109d35b2a3da => generated 0 bytes in 84 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 10:33:36.683944 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0810e37-6262-495f-b7f6-31896459f04d None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:33:36.684317 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0810e37-6262-495f-b7f6-31896459f04d None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:33:36.684649 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0810e37-6262-495f-b7f6-31896459f04d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:33:36.742330 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-c0810e37-6262-495f-b7f6-31896459f04d None None] MFA Rules not processed for user `37ce86ae50d84108b204e3c31b28f7d7`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:33:36.744035 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-c0810e37-6262-495f-b7f6-31896459f04d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:33:36.867648 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1108/2223] 10.222.0.49 () {58 vars in 957 bytes} [Wed Jun 15 10:33:36 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:33:36.879529 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259639fe-c788-45a4-8410-1de4722cb5a1 admin admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:33:36.881364 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-259639fe-c788-45a4-8410-1de4722cb5a1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:33:36.992196 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259639fe-c788-45a4-8410-1de4722cb5a1 admin admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:33:37.016644 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66342099-58a3-43b2-9873-63f181353f53 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91f2fe3be62e49548eae208e57bf2a93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '559b3e713f5844b6ae37151d56dd419f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:33:37.017561 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66342099-58a3-43b2-9873-63f181353f53 service glance] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:33:37.019192 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66342099-58a3-43b2-9873-63f181353f53 service glance] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:33:37.020288 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66342099-58a3-43b2-9873-63f181353f53 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:33:37.042651 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66342099-58a3-43b2-9873-63f181353f53 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:33:37.054534 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66342099-58a3-43b2-9873-63f181353f53 service glance] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:33:37.068761 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1116/2224] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Jun 15 10:33:36 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 190 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:33:37.363073 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69239f35-da64-4636-81fa-b4ba8b30e8d6 None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:33:37.364004 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69239f35-da64-4636-81fa-b4ba8b30e8d6 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:33:37.364977 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69239f35-da64-4636-81fa-b4ba8b30e8d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:33:37.391576 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-69239f35-da64-4636-81fa-b4ba8b30e8d6 None None] MFA Rules not processed for user `572ac94e242f414aab152286fa356335`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:33:37.393404 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-69239f35-da64-4636-81fa-b4ba8b30e8d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:33:37.414886 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:33:37.477591 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1109/2225] 10.222.0.49 () {58 vars in 957 bytes} [Wed Jun 15 10:33:37 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:33:38.316755 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd0b2dc-485e-4b18-9e0c-82697641ee37 None None] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:33:38.317803 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd0b2dc-485e-4b18-9e0c-82697641ee37 None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:33:38.318343 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd0b2dc-485e-4b18-9e0c-82697641ee37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:33:38.338469 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.auth.core [None req-acd0b2dc-485e-4b18-9e0c-82697641ee37 None None] MFA Rules not processed for user `103e10244eca444b86b9c51ff314d445`. Rule list: `[]` (Enabled: `True`). {{(pid=80119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:33:38.343365 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-acd0b2dc-485e-4b18-9e0c-82697641ee37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:33:38.409059 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1117/2226] 10.222.0.49 () {58 vars in 1009 bytes} [Wed Jun 15 10:33:38 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:33:38.421330 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69239f35-da64-4636-81fa-b4ba8b30e8d6 None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:33:38.422729 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-69239f35-da64-4636-81fa-b4ba8b30e8d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:33:38.507220 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69239f35-da64-4636-81fa-b4ba8b30e8d6 None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:33:38.512312 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381139ce-c73b-4c7a-affe-17a9ac9a80e3 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a69f99210444f9696e8b22615fb4a41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '559b3e713f5844b6ae37151d56dd419f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:33:38.513129 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381139ce-c73b-4c7a-affe-17a9ac9a80e3 service swift] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:33:38.513480 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381139ce-c73b-4c7a-affe-17a9ac9a80e3 service swift] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:33:38.513758 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381139ce-c73b-4c7a-affe-17a9ac9a80e3 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:33:38.523932 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-381139ce-c73b-4c7a-affe-17a9ac9a80e3 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:33:38.526890 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-381139ce-c73b-4c7a-affe-17a9ac9a80e3 service swift] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:33:38.537375 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1110/2227] 10.222.0.49 () {60 vars in 1384 bytes} [Wed Jun 15 10:33:38 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jun 15 10:33:39.927042 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a8755b-93d9-41cd-9720-1281ea98b25e None None] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:33:39.928349 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a8755b-93d9-41cd-9720-1281ea98b25e None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:33:39.928860 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a8755b-93d9-41cd-9720-1281ea98b25e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:33:39.950960 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.auth.core [None req-f8a8755b-93d9-41cd-9720-1281ea98b25e None None] MFA Rules not processed for user `6d152f74395c4b6cb889058f612d502f`. Rule list: `[]` (Enabled: `True`). {{(pid=80119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:33:39.952902 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-f8a8755b-93d9-41cd-9720-1281ea98b25e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:33:40.036698 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1118/2228] 10.222.0.49 () {58 vars in 957 bytes} [Wed Jun 15 10:33:39 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:33:48.533246 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=49 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:33:48.791925 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744deff2-9167-48c4-86b1-7a670c9f7079 None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:33:48.792208 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744deff2-9167-48c4-86b1-7a670c9f7079 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:33:48.792493 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744deff2-9167-48c4-86b1-7a670c9f7079 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:33:48.828337 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-744deff2-9167-48c4-86b1-7a670c9f7079 None None] MFA Rules not processed for user `103e10244eca444b86b9c51ff314d445`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:33:48.829862 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-744deff2-9167-48c4-86b1-7a670c9f7079 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:33:48.861835 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1111/2229] 10.222.0.49 () {58 vars in 1009 bytes} [Wed Jun 15 10:33:48 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 71 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:33:48.876498 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a8755b-93d9-41cd-9720-1281ea98b25e None None] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:33:48.917286 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a8755b-93d9-41cd-9720-1281ea98b25e None None] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:33:48.923796 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a409b3d0-b212-4f2d-937a-c2e05fbf7112 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a69f99210444f9696e8b22615fb4a41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '559b3e713f5844b6ae37151d56dd419f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:33:48.924465 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a409b3d0-b212-4f2d-937a-c2e05fbf7112 service swift] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:33:48.924752 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a409b3d0-b212-4f2d-937a-c2e05fbf7112 service swift] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:33:48.925031 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a409b3d0-b212-4f2d-937a-c2e05fbf7112 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:33:48.941240 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a409b3d0-b212-4f2d-937a-c2e05fbf7112 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:33:48.944302 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a409b3d0-b212-4f2d-937a-c2e05fbf7112 service swift] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:33:48.949464 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:33:48.958715 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1119/2230] 10.222.0.49 () {60 vars in 1384 bytes} [Wed Jun 15 10:33:48 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 15 10:33:58.859693 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=16 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:33:58.963931 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=2 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:34:06.566642 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744deff2-9167-48c4-86b1-7a670c9f7079 None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:34:06.658173 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744deff2-9167-48c4-86b1-7a670c9f7079 None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:34:06.664737 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5656222f-7107-4a70-936b-3d66441c6e8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:34:06.665618 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5656222f-7107-4a70-936b-3d66441c6e8e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:34:06.666286 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5656222f-7107-4a70-936b-3d66441c6e8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:34:06.666730 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5656222f-7107-4a70-936b-3d66441c6e8e admin admin] PATH_INFO: `/v3/users/37ce86ae50d84108b204e3c31b28f7d7` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:34:06.669249 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5656222f-7107-4a70-936b-3d66441c6e8e admin admin] RBAC: Authorizing `identity:delete_user(user_id=37ce86ae50d84108b204e3c31b28f7d7)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:34:06.671832 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5656222f-7107-4a70-936b-3d66441c6e8e admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:34:06.763768 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-5656222f-7107-4a70-936b-3d66441c6e8e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '37ce86ae50d84108b204e3c31b28f7d7'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:34:06.781723 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-5656222f-7107-4a70-936b-3d66441c6e8e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '37ce86ae50d84108b204e3c31b28f7d7'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:34:06.783128 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-5656222f-7107-4a70-936b-3d66441c6e8e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '37ce86ae50d84108b204e3c31b28f7d7'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:34:06.783747 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-5656222f-7107-4a70-936b-3d66441c6e8e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '37ce86ae50d84108b204e3c31b28f7d7'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:34:06.801676 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-5656222f-7107-4a70-936b-3d66441c6e8e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '37ce86ae50d84108b204e3c31b28f7d7'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:34:06.838957 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1112/2231] 10.222.0.49 () {58 vars in 1223 bytes} [Wed Jun 15 10:34:06 2022] DELETE /identity/v3/users/37ce86ae50d84108b204e3c31b28f7d7 => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:34:06.840601 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c92e8e8-d4bf-46c5-85d3-a53ed0c431b4 None None] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:34:06.841770 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c92e8e8-d4bf-46c5-85d3-a53ed0c431b4 None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:34:06.842399 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c92e8e8-d4bf-46c5-85d3-a53ed0c431b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:34:06.903283 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.auth.core [None req-3c92e8e8-d4bf-46c5-85d3-a53ed0c431b4 None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:34:06.905944 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-3c92e8e8-d4bf-46c5-85d3-a53ed0c431b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:34:06.994518 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1120/2232] 10.222.0.49 () {58 vars in 957 bytes} [Wed Jun 15 10:34:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:34:07.012640 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5656222f-7107-4a70-936b-3d66441c6e8e admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:34:07.014133 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-5656222f-7107-4a70-936b-3d66441c6e8e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:34:07.136945 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:34:07.153242 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5656222f-7107-4a70-936b-3d66441c6e8e admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:34:07.157639 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6b9ddf-820e-4a58-8882-7b8d3858faf8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0cb44a17d205431793e9a8791e477a32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '559b3e713f5844b6ae37151d56dd419f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:34:07.158303 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6b9ddf-820e-4a58-8882-7b8d3858faf8 service neutron] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:34:07.158588 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6b9ddf-820e-4a58-8882-7b8d3858faf8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:34:07.158865 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6b9ddf-820e-4a58-8882-7b8d3858faf8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:34:07.180130 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce6b9ddf-820e-4a58-8882-7b8d3858faf8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:34:07.181817 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce6b9ddf-820e-4a58-8882-7b8d3858faf8 service neutron] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:34:07.207178 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1113/2233] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Jun 15 10:34:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 192 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:34:07.466547 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5cc49f-3255-4f59-94fe-a0950083d5e8 None None] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:34:07.466943 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5cc49f-3255-4f59-94fe-a0950083d5e8 None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:34:07.467355 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5cc49f-3255-4f59-94fe-a0950083d5e8 None None] PATH_INFO: `/` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:34:07.468838 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1121/2234] 10.222.0.49 () {54 vars in 921 bytes} [Wed Jun 15 10:34:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:34:07.477006 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04c3127-9c6f-4e0d-8bc1-759ee2e1a580 None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:34:07.477298 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04c3127-9c6f-4e0d-8bc1-759ee2e1a580 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:34:07.477576 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04c3127-9c6f-4e0d-8bc1-759ee2e1a580 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:34:07.524000 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-b04c3127-9c6f-4e0d-8bc1-759ee2e1a580 None None] MFA Rules not processed for user `5ac54528370d4747b6e8777fba9d0c82`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:34:07.525482 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-b04c3127-9c6f-4e0d-8bc1-759ee2e1a580 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:34:07.591937 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5cc49f-3255-4f59-94fe-a0950083d5e8 None None] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:34:07.594883 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-7e5cc49f-3255-4f59-94fe-a0950083d5e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:34:07.608259 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1114/2235] 10.222.0.49 () {58 vars in 1018 bytes} [Wed Jun 15 10:34:07 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:34:07.621078 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04c3127-9c6f-4e0d-8bc1-759ee2e1a580 None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:34:07.621451 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5cc49f-3255-4f59-94fe-a0950083d5e8 None None] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:34:07.622776 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-b04c3127-9c6f-4e0d-8bc1-759ee2e1a580 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:34:07.627893 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e5fabc-30d0-4c6b-808f-1cd0d7b0eafb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:34:07.629365 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e5fabc-30d0-4c6b-808f-1cd0d7b0eafb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:34:07.629777 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e5fabc-30d0-4c6b-808f-1cd0d7b0eafb admin admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:34:07.631550 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e5fabc-30d0-4c6b-808f-1cd0d7b0eafb admin admin] PATH_INFO: `/v3/projects/2e616955fbdf473ea39c76678b402087` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:34:07.633082 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e5fabc-30d0-4c6b-808f-1cd0d7b0eafb admin admin] RBAC: Authorizing `identity:delete_project(project_id=2e616955fbdf473ea39c76678b402087)` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:34:07.636294 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e5fabc-30d0-4c6b-808f-1cd0d7b0eafb admin admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:34:07.729513 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-36e5fabc-30d0-4c6b-808f-1cd0d7b0eafb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2e616955fbdf473ea39c76678b402087', 'request_id': 'req-36e5fabc-30d0-4c6b-808f-1cd0d7b0eafb'} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:34:07.738190 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-36e5fabc-30d0-4c6b-808f-1cd0d7b0eafb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2e616955fbdf473ea39c76678b402087', 'request_id': 'req-36e5fabc-30d0-4c6b-808f-1cd0d7b0eafb'} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:34:07.746057 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04c3127-9c6f-4e0d-8bc1-759ee2e1a580 None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:34:07.746605 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-36e5fabc-30d0-4c6b-808f-1cd0d7b0eafb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2e616955fbdf473ea39c76678b402087', 'request_id': 'req-36e5fabc-30d0-4c6b-808f-1cd0d7b0eafb'} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:34:07.762866 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-36e5fabc-30d0-4c6b-808f-1cd0d7b0eafb admin admin] The token cache is being invalidate because project 2e616955fbdf473ea39c76678b402087 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80119) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 10:34:07.763433 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-36e5fabc-30d0-4c6b-808f-1cd0d7b0eafb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:34:07.764114 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-36e5fabc-30d0-4c6b-808f-1cd0d7b0eafb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:34:07.767164 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1122/2236] 10.222.0.49 () {58 vars in 1232 bytes} [Wed Jun 15 10:34:07 2022] DELETE /identity/v3/projects/2e616955fbdf473ea39c76678b402087 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 10:34:07.771709 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2b3aa8-9eb2-46f4-98ef-b0b421b73b69 None None] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:34:07.772198 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2b3aa8-9eb2-46f4-98ef-b0b421b73b69 None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:34:07.772625 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2b3aa8-9eb2-46f4-98ef-b0b421b73b69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:34:07.816563 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b553e0-c6ba-4ad1-af89-028bad453bb8 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91f2fe3be62e49548eae208e57bf2a93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '559b3e713f5844b6ae37151d56dd419f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:34:07.817512 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b553e0-c6ba-4ad1-af89-028bad453bb8 service glance] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:34:07.818410 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b553e0-c6ba-4ad1-af89-028bad453bb8 service glance] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:34:07.818887 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b553e0-c6ba-4ad1-af89-028bad453bb8 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:34:07.821418 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-41b553e0-c6ba-4ad1-af89-028bad453bb8 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:34:07.823738 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.auth.core [None req-dc2b3aa8-9eb2-46f4-98ef-b0b421b73b69 None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:34:07.825210 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-dc2b3aa8-9eb2-46f4-98ef-b0b421b73b69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:34:07.854247 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:34:07.874325 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:34:07.890453 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1123/2237] 10.222.0.49 () {58 vars in 957 bytes} [Wed Jun 15 10:34:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:34:07.899551 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d17553-b06f-430f-8454-cfa625bab840 None None] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:34:07.900050 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d17553-b06f-430f-8454-cfa625bab840 None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:34:07.900541 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d17553-b06f-430f-8454-cfa625bab840 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:34:07.900944 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b553e0-c6ba-4ad1-af89-028bad453bb8 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:34:07.902616 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b553e0-c6ba-4ad1-af89-028bad453bb8 service glance] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:34:07.915398 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1115/2238] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Jun 15 10:34:07 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:34:07.920464 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.auth.core [None req-21d17553-b06f-430f-8454-cfa625bab840 None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:34:07.921679 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-21d17553-b06f-430f-8454-cfa625bab840 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:34:07.930234 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1124/2239] 10.222.0.49 () {58 vars in 957 bytes} [Wed Jun 15 10:34:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 10:34:07.934061 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b553e0-c6ba-4ad1-af89-028bad453bb8 service glance] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:34:07.945043 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b553e0-c6ba-4ad1-af89-028bad453bb8 service glance] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:34:07.949766 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af93a2c-ab82-4387-b866-b1f2dd64b984 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:34:07.950429 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af93a2c-ab82-4387-b866-b1f2dd64b984 admin admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:34:07.950715 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af93a2c-ab82-4387-b866-b1f2dd64b984 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:34:07.950993 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af93a2c-ab82-4387-b866-b1f2dd64b984 admin admin] PATH_INFO: `/v3/domains` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:34:07.951779 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af93a2c-ab82-4387-b866-b1f2dd64b984 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:34:07.952086 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af93a2c-ab82-4387-b866-b1f2dd64b984 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:34:07.953825 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af93a2c-ab82-4387-b866-b1f2dd64b984 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:34:07.967883 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1116/2240] 10.222.0.49 () {58 vars in 1152 bytes} [Wed Jun 15 10:34:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:34:07.972343 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d17553-b06f-430f-8454-cfa625bab840 None None] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:34:07.982349 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d17553-b06f-430f-8454-cfa625bab840 None None] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:34:07.987358 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae071507-7d19-4575-adcf-a0321ed9ff5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:34:07.988024 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae071507-7d19-4575-adcf-a0321ed9ff5a admin admin] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:34:07.988312 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae071507-7d19-4575-adcf-a0321ed9ff5a admin admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:34:07.988591 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae071507-7d19-4575-adcf-a0321ed9ff5a admin admin] PATH_INFO: `/v3/projects` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:34:07.989276 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae071507-7d19-4575-adcf-a0321ed9ff5a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:34:07.991412 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae071507-7d19-4575-adcf-a0321ed9ff5a admin admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:34:07.991950 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38455ce7-06db-44d5-a170-a1847c31fa18 None None] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:34:07.992332 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38455ce7-06db-44d5-a170-a1847c31fa18 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:34:07.992615 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38455ce7-06db-44d5-a170-a1847c31fa18 None None] PATH_INFO: `/` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:34:07.993963 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1117/2241] 10.222.0.49 () {54 vars in 921 bytes} [Wed Jun 15 10:34:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 15 10:34:07.998418 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc136571-5905-4157-95a8-a8673c0faa74 None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:34:07.998742 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc136571-5905-4157-95a8-a8673c0faa74 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:34:07.999028 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc136571-5905-4157-95a8-a8673c0faa74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:34:08.020841 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1125/2242] 10.222.0.49 () {60 vars in 1152 bytes} [Wed Jun 15 10:34:07 2022] POST /identity/v3/projects => generated 375 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 10:34:08.021529 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-bc136571-5905-4157-95a8-a8673c0faa74 None None] MFA Rules not processed for user `0cb44a17d205431793e9a8791e477a32`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:34:08.021529 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-bc136571-5905-4157-95a8-a8673c0faa74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:34:08.022843 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae071507-7d19-4575-adcf-a0321ed9ff5a admin admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:34:08.075484 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1118/2243] 10.222.0.49 () {58 vars in 1018 bytes} [Wed Jun 15 10:34:07 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 10:34:08.088434 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc136571-5905-4157-95a8-a8673c0faa74 None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:34:08.090486 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-bc136571-5905-4157-95a8-a8673c0faa74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:34:08.092628 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae071507-7d19-4575-adcf-a0321ed9ff5a admin admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:34:08.098356 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206fc3ee-849a-4f23-be47-68f4ac7517b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:34:08.099937 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206fc3ee-849a-4f23-be47-68f4ac7517b3 admin admin] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:34:08.100461 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206fc3ee-849a-4f23-be47-68f4ac7517b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:34:08.100943 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206fc3ee-849a-4f23-be47-68f4ac7517b3 admin admin] PATH_INFO: `/v3/users` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:34:08.101877 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-206fc3ee-849a-4f23-be47-68f4ac7517b3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:34:08.102228 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc136571-5905-4157-95a8-a8673c0faa74 None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:34:08.104293 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-206fc3ee-849a-4f23-be47-68f4ac7517b3 admin admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:34:08.106166 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b4e6be-df03-44d7-8278-55799a7e9f53 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0cb44a17d205431793e9a8791e477a32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '559b3e713f5844b6ae37151d56dd419f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:34:08.106706 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b4e6be-df03-44d7-8278-55799a7e9f53 service neutron] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:34:08.106979 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b4e6be-df03-44d7-8278-55799a7e9f53 service neutron] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:34:08.107309 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b4e6be-df03-44d7-8278-55799a7e9f53 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:34:08.117023 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b4e6be-df03-44d7-8278-55799a7e9f53 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:34:08.118793 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b4e6be-df03-44d7-8278-55799a7e9f53 service neutron] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:34:08.128368 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1119/2244] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Jun 15 10:34:08 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 10:34:08.131932 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1126/2245] 10.222.0.49 () {60 vars in 1143 bytes} [Wed Jun 15 10:34:08 2022] POST /identity/v3/users => generated 288 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 10:34:08.139727 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b4e6be-df03-44d7-8278-55799a7e9f53 service neutron] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:34:08.152134 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b4e6be-df03-44d7-8278-55799a7e9f53 service neutron] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:34:08.157269 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3660ec-9567-4cb6-9a19-bcf8dffda8bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:34:08.157928 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3660ec-9567-4cb6-9a19-bcf8dffda8bb admin admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:34:08.158218 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3660ec-9567-4cb6-9a19-bcf8dffda8bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:34:08.158497 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3660ec-9567-4cb6-9a19-bcf8dffda8bb admin admin] PATH_INFO: `/v3/roles` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:34:08.159255 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3660ec-9567-4cb6-9a19-bcf8dffda8bb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:34:08.161024 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3660ec-9567-4cb6-9a19-bcf8dffda8bb admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:34:08.172283 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1120/2246] 10.222.0.49 () {58 vars in 1121 bytes} [Wed Jun 15 10:34:08 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 10:34:08.176347 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206fc3ee-849a-4f23-be47-68f4ac7517b3 admin admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:34:08.188585 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206fc3ee-849a-4f23-be47-68f4ac7517b3 admin admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:34:08.195888 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685a3635-0bca-411a-972d-05db2e701ba4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:34:08.196583 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685a3635-0bca-411a-972d-05db2e701ba4 admin admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:34:08.196873 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685a3635-0bca-411a-972d-05db2e701ba4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:34:08.197139 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685a3635-0bca-411a-972d-05db2e701ba4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:34:08.198106 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-685a3635-0bca-411a-972d-05db2e701ba4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:34:08.199691 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-685a3635-0bca-411a-972d-05db2e701ba4 admin admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:34:08.212343 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1127/2247] 10.222.0.49 () {58 vars in 1121 bytes} [Wed Jun 15 10:34:08 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 10:34:08.216573 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3660ec-9567-4cb6-9a19-bcf8dffda8bb admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:34:08.222203 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7261edac-15cd-4d96-9944-f86520759eff None None] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:34:08.222568 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7261edac-15cd-4d96-9944-f86520759eff None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:34:08.222943 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7261edac-15cd-4d96-9944-f86520759eff None None] PATH_INFO: `/` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:34:08.225135 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1128/2248] 10.222.0.49 () {54 vars in 921 bytes} [Wed Jun 15 10:34:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:34:08.228554 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3660ec-9567-4cb6-9a19-bcf8dffda8bb admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:34:08.234233 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f07f8cb-fd08-4711-8c5b-1eaaa0f1ff0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:34:08.235255 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f07f8cb-fd08-4711-8c5b-1eaaa0f1ff0b admin admin] REQUEST_METHOD: `PUT` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:34:08.235538 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f07f8cb-fd08-4711-8c5b-1eaaa0f1ff0b admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:34:08.235862 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f07f8cb-fd08-4711-8c5b-1eaaa0f1ff0b admin admin] PATH_INFO: `/v3/projects/eafe7837735844d8a6bbc3ec2ab8a484/users/669499e609e14cf39a69d2be08673a3c/roles/6876e57cc51c448097ab109d35b2a3da` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:34:08.236558 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8584210b-b770-4c72-a2ab-7b9eccd6035d None None] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:34:08.236854 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8584210b-b770-4c72-a2ab-7b9eccd6035d None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:34:08.237136 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8584210b-b770-4c72-a2ab-7b9eccd6035d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:34:08.256347 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f07f8cb-fd08-4711-8c5b-1eaaa0f1ff0b admin admin] RBAC: Authorizing `identity:create_grant(project_id=eafe7837735844d8a6bbc3ec2ab8a484, user_id=669499e609e14cf39a69d2be08673a3c, role_id=6876e57cc51c448097ab109d35b2a3da)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:34:08.260476 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f07f8cb-fd08-4711-8c5b-1eaaa0f1ff0b admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:34:08.266272 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.auth.core [None req-8584210b-b770-4c72-a2ab-7b9eccd6035d None None] MFA Rules not processed for user `5ac54528370d4747b6e8777fba9d0c82`. Rule list: `[]` (Enabled: `True`). {{(pid=80119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:34:08.267961 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-8584210b-b770-4c72-a2ab-7b9eccd6035d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:34:08.277335 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1121/2249] 10.222.0.49 () {60 vars in 1482 bytes} [Wed Jun 15 10:34:08 2022] PUT /identity/v3/projects/eafe7837735844d8a6bbc3ec2ab8a484/users/669499e609e14cf39a69d2be08673a3c/roles/6876e57cc51c448097ab109d35b2a3da => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 10:34:08.288084 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782a9dd3-b881-4830-ad78-f5a9299e38ad None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:34:08.288376 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782a9dd3-b881-4830-ad78-f5a9299e38ad None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:34:08.288654 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782a9dd3-b881-4830-ad78-f5a9299e38ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:34:08.318885 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-782a9dd3-b881-4830-ad78-f5a9299e38ad None None] MFA Rules not processed for user `669499e609e14cf39a69d2be08673a3c`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:34:08.319971 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-782a9dd3-b881-4830-ad78-f5a9299e38ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:34:08.358824 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1129/2250] 10.222.0.49 () {58 vars in 1018 bytes} [Wed Jun 15 10:34:08 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:34:08.371178 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8584210b-b770-4c72-a2ab-7b9eccd6035d None None] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:34:08.389558 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:34:08.404048 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1122/2251] 10.222.0.49 () {58 vars in 957 bytes} [Wed Jun 15 10:34:08 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 10:34:08.415218 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:34:08.417149 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782a9dd3-b881-4830-ad78-f5a9299e38ad None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:34:08.418712 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-782a9dd3-b881-4830-ad78-f5a9299e38ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:34:08.440350 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8584210b-b770-4c72-a2ab-7b9eccd6035d None None] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:34:08.444965 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649f7170-669e-4890-aa8f-97c1decf0283 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0cb44a17d205431793e9a8791e477a32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '559b3e713f5844b6ae37151d56dd419f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:34:08.445616 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649f7170-669e-4890-aa8f-97c1decf0283 service neutron] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:34:08.445902 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649f7170-669e-4890-aa8f-97c1decf0283 service neutron] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:34:08.446185 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649f7170-669e-4890-aa8f-97c1decf0283 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:34:08.456362 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649f7170-669e-4890-aa8f-97c1decf0283 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:34:08.457794 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649f7170-669e-4890-aa8f-97c1decf0283 service neutron] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:34:08.469196 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1130/2252] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Jun 15 10:34:08 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:34:08.479326 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782a9dd3-b881-4830-ad78-f5a9299e38ad None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:34:08.482964 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5279572-344e-4626-9204-4f9a850bb45b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20d1327553404b80aecee87c78fcb52b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '559b3e713f5844b6ae37151d56dd419f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:34:08.483643 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5279572-344e-4626-9204-4f9a850bb45b service cinder] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:34:08.483926 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5279572-344e-4626-9204-4f9a850bb45b service cinder] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:34:08.484201 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5279572-344e-4626-9204-4f9a850bb45b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:34:08.493206 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5279572-344e-4626-9204-4f9a850bb45b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:34:08.494644 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5279572-344e-4626-9204-4f9a850bb45b service cinder] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:34:08.503748 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1123/2253] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Jun 15 10:34:08 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 10:34:18.464863 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=11 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:34:18.499541 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=29 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:34:18.601166 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649f7170-669e-4890-aa8f-97c1decf0283 service neutron] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:34:18.668990 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649f7170-669e-4890-aa8f-97c1decf0283 service neutron] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:34:18.673713 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2852a67b-255b-4adb-b08e-d46ea10bac85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:34:18.674608 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2852a67b-255b-4adb-b08e-d46ea10bac85 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:34:18.674897 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2852a67b-255b-4adb-b08e-d46ea10bac85 admin admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:34:18.675213 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2852a67b-255b-4adb-b08e-d46ea10bac85 admin admin] PATH_INFO: `/v3/users/669499e609e14cf39a69d2be08673a3c` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:34:18.676808 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2852a67b-255b-4adb-b08e-d46ea10bac85 admin admin] RBAC: Authorizing `identity:delete_user(user_id=669499e609e14cf39a69d2be08673a3c)` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:34:18.679505 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2852a67b-255b-4adb-b08e-d46ea10bac85 admin admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:34:18.731389 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-2852a67b-255b-4adb-b08e-d46ea10bac85 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '669499e609e14cf39a69d2be08673a3c'} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:34:18.738750 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-2852a67b-255b-4adb-b08e-d46ea10bac85 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '669499e609e14cf39a69d2be08673a3c'} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:34:18.739934 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-2852a67b-255b-4adb-b08e-d46ea10bac85 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '669499e609e14cf39a69d2be08673a3c'} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:34:18.741210 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-2852a67b-255b-4adb-b08e-d46ea10bac85 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '669499e609e14cf39a69d2be08673a3c'} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:34:18.749028 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-2852a67b-255b-4adb-b08e-d46ea10bac85 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '669499e609e14cf39a69d2be08673a3c'} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:34:18.767847 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1131/2254] 10.222.0.49 () {58 vars in 1223 bytes} [Wed Jun 15 10:34:18 2022] DELETE /identity/v3/users/669499e609e14cf39a69d2be08673a3c => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:34:18.772579 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554ddf9c-beaa-42ac-ae9d-635c2ec6d5bb None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:34:18.773147 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554ddf9c-beaa-42ac-ae9d-635c2ec6d5bb None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:34:18.773637 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554ddf9c-beaa-42ac-ae9d-635c2ec6d5bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:34:18.797828 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-554ddf9c-beaa-42ac-ae9d-635c2ec6d5bb None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:34:18.799079 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-554ddf9c-beaa-42ac-ae9d-635c2ec6d5bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:34:18.863280 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1124/2255] 10.222.0.49 () {58 vars in 957 bytes} [Wed Jun 15 10:34:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:34:18.873484 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2852a67b-255b-4adb-b08e-d46ea10bac85 admin admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:34:18.875188 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-2852a67b-255b-4adb-b08e-d46ea10bac85 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:34:18.933826 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2852a67b-255b-4adb-b08e-d46ea10bac85 admin admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:34:18.937648 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63380b22-92b8-49ee-98a4-dfbbd32ff424 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0cb44a17d205431793e9a8791e477a32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '559b3e713f5844b6ae37151d56dd419f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:34:18.938367 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63380b22-92b8-49ee-98a4-dfbbd32ff424 service neutron] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:34:18.938656 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63380b22-92b8-49ee-98a4-dfbbd32ff424 service neutron] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:34:18.938933 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63380b22-92b8-49ee-98a4-dfbbd32ff424 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:34:18.949009 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63380b22-92b8-49ee-98a4-dfbbd32ff424 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:34:18.950444 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63380b22-92b8-49ee-98a4-dfbbd32ff424 service neutron] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:34:18.960990 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1132/2256] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Jun 15 10:34:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:34:19.232062 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554ddf9c-beaa-42ac-ae9d-635c2ec6d5bb None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:34:19.233519 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-554ddf9c-beaa-42ac-ae9d-635c2ec6d5bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:34:19.245985 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554ddf9c-beaa-42ac-ae9d-635c2ec6d5bb None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:34:19.250791 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a5706c-4095-411a-ad8b-c8b029856661 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:34:19.251741 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a5706c-4095-411a-ad8b-c8b029856661 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:34:19.252029 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a5706c-4095-411a-ad8b-c8b029856661 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:34:19.252472 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a5706c-4095-411a-ad8b-c8b029856661 admin admin] PATH_INFO: `/v3/projects/eafe7837735844d8a6bbc3ec2ab8a484` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:34:19.253647 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a5706c-4095-411a-ad8b-c8b029856661 admin admin] RBAC: Authorizing `identity:delete_project(project_id=eafe7837735844d8a6bbc3ec2ab8a484)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:34:19.255680 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a5706c-4095-411a-ad8b-c8b029856661 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:34:19.310521 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-c8a5706c-4095-411a-ad8b-c8b029856661 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eafe7837735844d8a6bbc3ec2ab8a484', 'request_id': 'req-c8a5706c-4095-411a-ad8b-c8b029856661'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:34:19.319324 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-c8a5706c-4095-411a-ad8b-c8b029856661 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eafe7837735844d8a6bbc3ec2ab8a484', 'request_id': 'req-c8a5706c-4095-411a-ad8b-c8b029856661'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:34:19.326099 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-c8a5706c-4095-411a-ad8b-c8b029856661 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eafe7837735844d8a6bbc3ec2ab8a484', 'request_id': 'req-c8a5706c-4095-411a-ad8b-c8b029856661'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:34:19.341266 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-c8a5706c-4095-411a-ad8b-c8b029856661 admin admin] The token cache is being invalidate because project eafe7837735844d8a6bbc3ec2ab8a484 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80120) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 10:34:19.341715 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-c8a5706c-4095-411a-ad8b-c8b029856661 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:34:19.342574 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-c8a5706c-4095-411a-ad8b-c8b029856661 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:34:19.344403 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1125/2257] 10.222.0.49 () {58 vars in 1232 bytes} [Wed Jun 15 10:34:19 2022] DELETE /identity/v3/projects/eafe7837735844d8a6bbc3ec2ab8a484 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:34:21.049419 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d0c67a-6f7b-4da2-a979-70a2b34251bc None None] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:34:21.049827 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d0c67a-6f7b-4da2-a979-70a2b34251bc None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:34:21.050141 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d0c67a-6f7b-4da2-a979-70a2b34251bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:34:21.069153 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.auth.core [None req-e1d0c67a-6f7b-4da2-a979-70a2b34251bc None None] MFA Rules not processed for user `103e10244eca444b86b9c51ff314d445`. Rule list: `[]` (Enabled: `True`). {{(pid=80119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:34:21.070416 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-e1d0c67a-6f7b-4da2-a979-70a2b34251bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:34:21.089242 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:34:21.140462 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1133/2258] 10.222.0.49 () {58 vars in 1009 bytes} [Wed Jun 15 10:34:21 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:34:21.153804 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a5706c-4095-411a-ad8b-c8b029856661 admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:34:21.156040 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-c8a5706c-4095-411a-ad8b-c8b029856661 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:34:21.284876 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a5706c-4095-411a-ad8b-c8b029856661 admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:34:21.295106 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1914a2e2-92e5-465d-99a3-6cf0d9dec3b6 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a69f99210444f9696e8b22615fb4a41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '559b3e713f5844b6ae37151d56dd419f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:34:21.296383 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1914a2e2-92e5-465d-99a3-6cf0d9dec3b6 service swift] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:34:21.296896 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1914a2e2-92e5-465d-99a3-6cf0d9dec3b6 service swift] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:34:21.297346 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1914a2e2-92e5-465d-99a3-6cf0d9dec3b6 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:34:21.306846 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1914a2e2-92e5-465d-99a3-6cf0d9dec3b6 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:34:21.308487 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1914a2e2-92e5-465d-99a3-6cf0d9dec3b6 service swift] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:34:21.325710 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1126/2259] 10.222.0.49 () {60 vars in 1384 bytes} [Wed Jun 15 10:34:21 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 15 10:34:31.134416 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=18 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:34:31.318205 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=86,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:34:32.348776 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d0c67a-6f7b-4da2-a979-70a2b34251bc None None] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:34:32.350244 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-e1d0c67a-6f7b-4da2-a979-70a2b34251bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:34:32.448237 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d0c67a-6f7b-4da2-a979-70a2b34251bc None None] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:34:32.453312 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bae0aa1-f713-4dbe-98c7-bac56c1d9b51 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd03612a525644518517069b3fa43522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '559b3e713f5844b6ae37151d56dd419f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:34:32.453960 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bae0aa1-f713-4dbe-98c7-bac56c1d9b51 service placement] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:34:32.454247 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bae0aa1-f713-4dbe-98c7-bac56c1d9b51 service placement] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:34:32.454525 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bae0aa1-f713-4dbe-98c7-bac56c1d9b51 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:34:32.456303 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-2bae0aa1-f713-4dbe-98c7-bac56c1d9b51 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:34:32.467787 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bae0aa1-f713-4dbe-98c7-bac56c1d9b51 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:34:32.469365 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bae0aa1-f713-4dbe-98c7-bac56c1d9b51 service placement] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:34:32.479155 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1134/2260] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Jun 15 10:34:32 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:34:36.299742 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1914a2e2-92e5-465d-99a3-6cf0d9dec3b6 service swift] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:34:36.301527 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-1914a2e2-92e5-465d-99a3-6cf0d9dec3b6 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:34:36.382836 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1914a2e2-92e5-465d-99a3-6cf0d9dec3b6 service swift] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:34:36.388756 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c53ac9-b245-49ca-8da6-d871e29e1d46 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0cb44a17d205431793e9a8791e477a32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '559b3e713f5844b6ae37151d56dd419f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:34:36.389480 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c53ac9-b245-49ca-8da6-d871e29e1d46 service neutron] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:34:36.389833 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c53ac9-b245-49ca-8da6-d871e29e1d46 service neutron] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:34:36.390305 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c53ac9-b245-49ca-8da6-d871e29e1d46 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:34:36.396267 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-09c53ac9-b245-49ca-8da6-d871e29e1d46 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:34:36.460189 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c53ac9-b245-49ca-8da6-d871e29e1d46 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:34:36.462164 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c53ac9-b245-49ca-8da6-d871e29e1d46 service neutron] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:34:36.471526 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1127/2261] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Jun 15 10:34:36 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:34:42.475065 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=41 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:34:42.956045 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359e2943-f0fc-4441-ba77-7652ca0a0aa7 None None] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:34:42.956338 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359e2943-f0fc-4441-ba77-7652ca0a0aa7 None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:34:42.956617 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359e2943-f0fc-4441-ba77-7652ca0a0aa7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:34:43.059557 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.auth.core [None req-359e2943-f0fc-4441-ba77-7652ca0a0aa7 None None] MFA Rules not processed for user `103e10244eca444b86b9c51ff314d445`. Rule list: `[]` (Enabled: `True`). {{(pid=80119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:34:43.061014 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-359e2943-f0fc-4441-ba77-7652ca0a0aa7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:34:43.089857 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1135/2262] 10.222.0.49 () {58 vars in 967 bytes} [Wed Jun 15 10:34:42 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:34:43.119908 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c53ac9-b245-49ca-8da6-d871e29e1d46 service neutron] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:34:43.163869 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c53ac9-b245-49ca-8da6-d871e29e1d46 service neutron] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:34:43.170270 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61d743a-79a1-4692-a8d9-7f648f17a644 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a69f99210444f9696e8b22615fb4a41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '559b3e713f5844b6ae37151d56dd419f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:34:43.171195 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61d743a-79a1-4692-a8d9-7f648f17a644 service swift] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:34:43.171726 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61d743a-79a1-4692-a8d9-7f648f17a644 service swift] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:34:43.172197 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61d743a-79a1-4692-a8d9-7f648f17a644 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:34:43.183555 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61d743a-79a1-4692-a8d9-7f648f17a644 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:34:43.185467 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61d743a-79a1-4692-a8d9-7f648f17a644 service swift] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:34:43.196784 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1128/2263] 10.222.0.49 () {60 vars in 1384 bytes} [Wed Jun 15 10:34:43 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 77 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 15 10:34:53.077181 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=18 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:34:53.192704 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=60 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:34:55.292666 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43b7f15-aab4-4253-9a1f-9f9309b3cfb6 None None] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:34:55.293192 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43b7f15-aab4-4253-9a1f-9f9309b3cfb6 None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:34:55.293297 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43b7f15-aab4-4253-9a1f-9f9309b3cfb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:34:55.312150 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.auth.core [None req-c43b7f15-aab4-4253-9a1f-9f9309b3cfb6 None None] MFA Rules not processed for user `14b720a2d0854d218272390205887100`. Rule list: `[]` (Enabled: `True`). {{(pid=80119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:34:55.313362 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-c43b7f15-aab4-4253-9a1f-9f9309b3cfb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:34:55.367217 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1136/2264] 10.222.0.49 () {58 vars in 957 bytes} [Wed Jun 15 10:34:55 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:35:05.360413 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=25 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:35:34.235231 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2d91b2-0be6-4b32-8f9c-02b5472015b9 None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:35:34.235770 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2d91b2-0be6-4b32-8f9c-02b5472015b9 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:35:34.235770 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2d91b2-0be6-4b32-8f9c-02b5472015b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:35:34.253552 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-ae2d91b2-0be6-4b32-8f9c-02b5472015b9 None None] MFA Rules not processed for user `cca7cb5e7620446ebae91b1fe12bbd24`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:35:34.254754 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-ae2d91b2-0be6-4b32-8f9c-02b5472015b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:35:34.306423 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1129/2265] 10.222.0.49 () {58 vars in 957 bytes} [Wed Jun 15 10:35:34 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 72 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:35:36.460101 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de803a19-c8a4-4baa-8d2e-127677d9666c None None] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:35:36.460627 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de803a19-c8a4-4baa-8d2e-127677d9666c None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:35:36.460627 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de803a19-c8a4-4baa-8d2e-127677d9666c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:35:36.478406 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.auth.core [None req-de803a19-c8a4-4baa-8d2e-127677d9666c None None] MFA Rules not processed for user `6d152f74395c4b6cb889058f612d502f`. Rule list: `[]` (Enabled: `True`). {{(pid=80119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:35:36.479644 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-de803a19-c8a4-4baa-8d2e-127677d9666c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:35:36.532588 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1137/2266] 10.222.0.49 () {58 vars in 957 bytes} [Wed Jun 15 10:35:36 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:35:44.300061 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=25 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:35:46.525461 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=25 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:36:14.474016 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2dfd53-9bf9-4706-ac0f-6fd0b049de34 None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:36:14.474540 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2dfd53-9bf9-4706-ac0f-6fd0b049de34 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:36:14.474540 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2dfd53-9bf9-4706-ac0f-6fd0b049de34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:36:14.492493 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-9b2dfd53-9bf9-4706-ac0f-6fd0b049de34 None None] MFA Rules not processed for user `103e10244eca444b86b9c51ff314d445`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:36:14.493708 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-9b2dfd53-9bf9-4706-ac0f-6fd0b049de34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:36:14.500862 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1130/2267] 10.222.0.49 () {58 vars in 967 bytes} [Wed Jun 15 10:36:14 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:36:14.552524 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de803a19-c8a4-4baa-8d2e-127677d9666c None None] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:36:14.566022 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de803a19-c8a4-4baa-8d2e-127677d9666c None None] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:36:14.571364 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6f8b3d-fecd-414f-8cd7-5485a789d3aa service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a69f99210444f9696e8b22615fb4a41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '559b3e713f5844b6ae37151d56dd419f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:36:14.572104 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6f8b3d-fecd-414f-8cd7-5485a789d3aa service swift] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:36:14.572455 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6f8b3d-fecd-414f-8cd7-5485a789d3aa service swift] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:36:14.572820 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6f8b3d-fecd-414f-8cd7-5485a789d3aa service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:36:14.581702 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6f8b3d-fecd-414f-8cd7-5485a789d3aa service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:36:14.583408 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6f8b3d-fecd-414f-8cd7-5485a789d3aa service swift] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:36:14.591600 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1138/2268] 10.222.0.49 () {60 vars in 1384 bytes} [Wed Jun 15 10:36:14 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 15 10:36:15.125158 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cea9d6f-a72d-4610-8985-d64c906df6d2 None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:36:15.125533 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cea9d6f-a72d-4610-8985-d64c906df6d2 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:36:15.125900 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cea9d6f-a72d-4610-8985-d64c906df6d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:36:15.145315 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-6cea9d6f-a72d-4610-8985-d64c906df6d2 None None] MFA Rules not processed for user `14b720a2d0854d218272390205887100`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:36:15.146729 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-6cea9d6f-a72d-4610-8985-d64c906df6d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:36:15.154534 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1131/2269] 10.222.0.49 () {58 vars in 957 bytes} [Wed Jun 15 10:36:15 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:36:16.165509 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6f8b3d-fecd-414f-8cd7-5485a789d3aa service swift] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:36:16.167103 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-2a6f8b3d-fecd-414f-8cd7-5485a789d3aa service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:36:16.181684 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6f8b3d-fecd-414f-8cd7-5485a789d3aa service swift] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:36:16.185855 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3357eae7-1461-45bd-8603-fbc4e7d08ada service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0cb44a17d205431793e9a8791e477a32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '559b3e713f5844b6ae37151d56dd419f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:36:16.186802 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3357eae7-1461-45bd-8603-fbc4e7d08ada service neutron] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:36:16.187161 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3357eae7-1461-45bd-8603-fbc4e7d08ada service neutron] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:36:16.187405 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3357eae7-1461-45bd-8603-fbc4e7d08ada service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:36:16.189905 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-3357eae7-1461-45bd-8603-fbc4e7d08ada service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:36:16.305757 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3357eae7-1461-45bd-8603-fbc4e7d08ada service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:36:16.307263 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3357eae7-1461-45bd-8603-fbc4e7d08ada service neutron] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:36:16.342669 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1139/2270] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Jun 15 10:36:16 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 177 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:36:17.506872 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cea9d6f-a72d-4610-8985-d64c906df6d2 None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:36:17.519224 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cea9d6f-a72d-4610-8985-d64c906df6d2 None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:36:17.523147 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ddbe24-69de-4c6d-93da-9fb5216c97ac service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0cb44a17d205431793e9a8791e477a32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '559b3e713f5844b6ae37151d56dd419f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:36:17.523795 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ddbe24-69de-4c6d-93da-9fb5216c97ac service neutron] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:36:17.524234 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ddbe24-69de-4c6d-93da-9fb5216c97ac service neutron] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:36:17.524669 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ddbe24-69de-4c6d-93da-9fb5216c97ac service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:36:17.527891 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-b2ddbe24-69de-4c6d-93da-9fb5216c97ac service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:36:17.546393 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ddbe24-69de-4c6d-93da-9fb5216c97ac service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:36:17.548567 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ddbe24-69de-4c6d-93da-9fb5216c97ac service neutron] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:36:17.564581 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1132/2271] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Jun 15 10:36:17 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:36:23.148676 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3357eae7-1461-45bd-8603-fbc4e7d08ada service neutron] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:36:23.151252 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-3357eae7-1461-45bd-8603-fbc4e7d08ada service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:36:23.233523 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3357eae7-1461-45bd-8603-fbc4e7d08ada service neutron] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:36:23.238130 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6954cf3e-0563-436d-823b-a02ee54b6236 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:36:23.239012 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6954cf3e-0563-436d-823b-a02ee54b6236 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:36:23.239331 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6954cf3e-0563-436d-823b-a02ee54b6236 admin admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:36:23.239610 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6954cf3e-0563-436d-823b-a02ee54b6236 admin admin] PATH_INFO: `/v3/users/572ac94e242f414aab152286fa356335` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:36:23.240827 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6954cf3e-0563-436d-823b-a02ee54b6236 admin admin] RBAC: Authorizing `identity:delete_user(user_id=572ac94e242f414aab152286fa356335)` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:36:23.242783 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6954cf3e-0563-436d-823b-a02ee54b6236 admin admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:36:23.290317 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-6954cf3e-0563-436d-823b-a02ee54b6236 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '572ac94e242f414aab152286fa356335'} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:36:23.295522 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:36:23.297774 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-6954cf3e-0563-436d-823b-a02ee54b6236 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '572ac94e242f414aab152286fa356335'} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:36:23.298544 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-6954cf3e-0563-436d-823b-a02ee54b6236 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '572ac94e242f414aab152286fa356335'} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:36:23.300003 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-6954cf3e-0563-436d-823b-a02ee54b6236 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '572ac94e242f414aab152286fa356335'} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:36:23.307175 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-6954cf3e-0563-436d-823b-a02ee54b6236 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '572ac94e242f414aab152286fa356335'} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:36:23.324708 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1140/2272] 10.222.0.49 () {58 vars in 1223 bytes} [Wed Jun 15 10:36:23 2022] DELETE /identity/v3/users/572ac94e242f414aab152286fa356335 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:36:23.502136 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ddbe24-69de-4c6d-93da-9fb5216c97ac service neutron] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:36:23.516320 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ddbe24-69de-4c6d-93da-9fb5216c97ac service neutron] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:36:23.640338 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cee227f-32be-4213-9a80-b43155d80efd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:36:23.641841 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cee227f-32be-4213-9a80-b43155d80efd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:36:23.642398 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cee227f-32be-4213-9a80-b43155d80efd admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:36:23.642957 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cee227f-32be-4213-9a80-b43155d80efd admin admin] PATH_INFO: `/v3/projects/440f69d2725e435ab11b2fc658e1e745` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:36:23.644959 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cee227f-32be-4213-9a80-b43155d80efd admin admin] RBAC: Authorizing `identity:delete_project(project_id=440f69d2725e435ab11b2fc658e1e745)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:36:23.648497 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cee227f-32be-4213-9a80-b43155d80efd admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:36:23.649695 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6954cf3e-0563-436d-823b-a02ee54b6236 admin admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:36:23.651824 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-6954cf3e-0563-436d-823b-a02ee54b6236 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:36:23.748537 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-2cee227f-32be-4213-9a80-b43155d80efd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '440f69d2725e435ab11b2fc658e1e745', 'request_id': 'req-2cee227f-32be-4213-9a80-b43155d80efd'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:36:23.759380 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-2cee227f-32be-4213-9a80-b43155d80efd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '440f69d2725e435ab11b2fc658e1e745', 'request_id': 'req-2cee227f-32be-4213-9a80-b43155d80efd'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:36:23.768040 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-2cee227f-32be-4213-9a80-b43155d80efd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '440f69d2725e435ab11b2fc658e1e745', 'request_id': 'req-2cee227f-32be-4213-9a80-b43155d80efd'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:36:23.776857 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6954cf3e-0563-436d-823b-a02ee54b6236 admin admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:36:23.782721 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-2cee227f-32be-4213-9a80-b43155d80efd admin admin] The token cache is being invalidate because project 440f69d2725e435ab11b2fc658e1e745 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80120) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 10:36:23.783398 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-2cee227f-32be-4213-9a80-b43155d80efd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:36:23.783932 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-2cee227f-32be-4213-9a80-b43155d80efd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:36:23.786319 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1133/2273] 10.222.0.49 () {58 vars in 1232 bytes} [Wed Jun 15 10:36:23 2022] DELETE /identity/v3/projects/440f69d2725e435ab11b2fc658e1e745 => generated 0 bytes in 284 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:36:23.791585 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cee227f-32be-4213-9a80-b43155d80efd admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:36:23.793146 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-2cee227f-32be-4213-9a80-b43155d80efd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:36:23.862073 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a7ba50-8fdd-47ef-a2e7-34291ca7704d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20d1327553404b80aecee87c78fcb52b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '559b3e713f5844b6ae37151d56dd419f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:36:23.863217 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a7ba50-8fdd-47ef-a2e7-34291ca7704d service cinder] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:36:23.863661 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a7ba50-8fdd-47ef-a2e7-34291ca7704d service cinder] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:36:23.864484 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a7ba50-8fdd-47ef-a2e7-34291ca7704d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:36:23.866519 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-91a7ba50-8fdd-47ef-a2e7-34291ca7704d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:36:23.901510 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cee227f-32be-4213-9a80-b43155d80efd admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:36:23.907272 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f292b4-766f-4c16-ae7e-10e456d8003e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:36:23.908233 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f292b4-766f-4c16-ae7e-10e456d8003e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:36:23.908590 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f292b4-766f-4c16-ae7e-10e456d8003e admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:36:23.908940 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f292b4-766f-4c16-ae7e-10e456d8003e admin admin] PATH_INFO: `/v3/users/412744e697f94346aeeabe2a1422aa53` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:36:23.910240 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f292b4-766f-4c16-ae7e-10e456d8003e admin admin] RBAC: Authorizing `identity:delete_user(user_id=412744e697f94346aeeabe2a1422aa53)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:36:23.912353 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f292b4-766f-4c16-ae7e-10e456d8003e admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:36:23.918947 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:36:23.932228 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a7ba50-8fdd-47ef-a2e7-34291ca7704d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:36:23.934038 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a7ba50-8fdd-47ef-a2e7-34291ca7704d service cinder] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:36:23.944153 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1141/2274] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Jun 15 10:36:23 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:36:23.987622 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-16f292b4-766f-4c16-ae7e-10e456d8003e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '412744e697f94346aeeabe2a1422aa53'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:36:23.997040 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-16f292b4-766f-4c16-ae7e-10e456d8003e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '412744e697f94346aeeabe2a1422aa53'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:36:23.997412 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-16f292b4-766f-4c16-ae7e-10e456d8003e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '412744e697f94346aeeabe2a1422aa53'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:36:23.998192 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-16f292b4-766f-4c16-ae7e-10e456d8003e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '412744e697f94346aeeabe2a1422aa53'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:36:24.008216 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-16f292b4-766f-4c16-ae7e-10e456d8003e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '412744e697f94346aeeabe2a1422aa53'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:36:24.031929 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1134/2275] 10.222.0.49 () {58 vars in 1223 bytes} [Wed Jun 15 10:36:23 2022] DELETE /identity/v3/users/412744e697f94346aeeabe2a1422aa53 => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:36:24.204369 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a7ba50-8fdd-47ef-a2e7-34291ca7704d service cinder] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:36:24.217119 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a7ba50-8fdd-47ef-a2e7-34291ca7704d service cinder] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:36:24.240554 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:36:24.285475 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b83ad47-af06-4107-a411-c18807f1f528 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:36:24.286349 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b83ad47-af06-4107-a411-c18807f1f528 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:36:24.286635 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b83ad47-af06-4107-a411-c18807f1f528 admin admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:36:24.286913 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b83ad47-af06-4107-a411-c18807f1f528 admin admin] PATH_INFO: `/v3/projects/ed7082a13129477c8ff6630ced935884` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:36:24.288129 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b83ad47-af06-4107-a411-c18807f1f528 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ed7082a13129477c8ff6630ced935884)` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:36:24.290070 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b83ad47-af06-4107-a411-c18807f1f528 admin admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:36:24.337435 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-9b83ad47-af06-4107-a411-c18807f1f528 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ed7082a13129477c8ff6630ced935884', 'request_id': 'req-9b83ad47-af06-4107-a411-c18807f1f528'} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:36:24.345015 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-9b83ad47-af06-4107-a411-c18807f1f528 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ed7082a13129477c8ff6630ced935884', 'request_id': 'req-9b83ad47-af06-4107-a411-c18807f1f528'} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:36:24.350453 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-9b83ad47-af06-4107-a411-c18807f1f528 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ed7082a13129477c8ff6630ced935884', 'request_id': 'req-9b83ad47-af06-4107-a411-c18807f1f528'} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:36:24.361732 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-9b83ad47-af06-4107-a411-c18807f1f528 admin admin] The token cache is being invalidate because project ed7082a13129477c8ff6630ced935884 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80119) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 10:36:24.362009 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-9b83ad47-af06-4107-a411-c18807f1f528 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:36:24.362653 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-9b83ad47-af06-4107-a411-c18807f1f528 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:36:24.371218 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1142/2276] 10.222.0.49 () {58 vars in 1232 bytes} [Wed Jun 15 10:36:24 2022] DELETE /identity/v3/projects/ed7082a13129477c8ff6630ced935884 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:36:32.887278 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f292b4-766f-4c16-ae7e-10e456d8003e admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:36:32.889691 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-16f292b4-766f-4c16-ae7e-10e456d8003e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:36:32.962788 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f292b4-766f-4c16-ae7e-10e456d8003e admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:36:33.026721 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df90249-54c6-4e28-915a-64efd5d1aa29 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0cb44a17d205431793e9a8791e477a32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '559b3e713f5844b6ae37151d56dd419f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:36:33.027697 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df90249-54c6-4e28-915a-64efd5d1aa29 service neutron] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:36:33.027980 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df90249-54c6-4e28-915a-64efd5d1aa29 service neutron] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:36:33.028250 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df90249-54c6-4e28-915a-64efd5d1aa29 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:36:33.030454 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-9df90249-54c6-4e28-915a-64efd5d1aa29 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:36:33.040828 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df90249-54c6-4e28-915a-64efd5d1aa29 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:36:33.042436 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df90249-54c6-4e28-915a-64efd5d1aa29 service neutron] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:36:33.053041 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1135/2277] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Jun 15 10:36:32 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:36:34.358220 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=41,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:36:36.259018 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b83ad47-af06-4107-a411-c18807f1f528 admin admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:36:36.260442 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-9b83ad47-af06-4107-a411-c18807f1f528 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:36:36.339904 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b83ad47-af06-4107-a411-c18807f1f528 admin admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:36:36.346276 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e47278-f616-4399-9b8a-7acfb8899620 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ac54528370d4747b6e8777fba9d0c82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '559b3e713f5844b6ae37151d56dd419f', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:36:36.347038 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e47278-f616-4399-9b8a-7acfb8899620 service nova] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:36:36.347613 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e47278-f616-4399-9b8a-7acfb8899620 service nova] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:36:36.347863 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e47278-f616-4399-9b8a-7acfb8899620 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:36:36.349834 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-e4e47278-f616-4399-9b8a-7acfb8899620 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:36:36.408315 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e47278-f616-4399-9b8a-7acfb8899620 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:36:36.409913 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e47278-f616-4399-9b8a-7acfb8899620 service nova] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:36:36.418870 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1143/2278] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Jun 15 10:36:36 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:36:39.871684 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df90249-54c6-4e28-915a-64efd5d1aa29 service neutron] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:36:39.873138 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-9df90249-54c6-4e28-915a-64efd5d1aa29 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:36:39.904030 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df90249-54c6-4e28-915a-64efd5d1aa29 service neutron] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:36:39.911715 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4bd2bb-ffd0-467c-ae94-5d470aeb8b0c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ac54528370d4747b6e8777fba9d0c82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '559b3e713f5844b6ae37151d56dd419f', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:36:39.912382 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4bd2bb-ffd0-467c-ae94-5d470aeb8b0c service nova] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:36:39.914684 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4bd2bb-ffd0-467c-ae94-5d470aeb8b0c service nova] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:36:39.914684 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4bd2bb-ffd0-467c-ae94-5d470aeb8b0c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:36:39.914895 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-8d4bd2bb-ffd0-467c-ae94-5d470aeb8b0c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:36:39.927754 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d4bd2bb-ffd0-467c-ae94-5d470aeb8b0c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:36:39.929346 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d4bd2bb-ffd0-467c-ae94-5d470aeb8b0c service nova] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:36:39.952192 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1136/2279] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Jun 15 10:36:39 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:36:43.663491 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e07d35-a106-4eff-b66a-1eda46682d3d None None] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:36:43.664016 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e07d35-a106-4eff-b66a-1eda46682d3d None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:36:43.664125 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e07d35-a106-4eff-b66a-1eda46682d3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:36:43.682243 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.auth.core [None req-84e07d35-a106-4eff-b66a-1eda46682d3d None None] MFA Rules not processed for user `9ca22364db8147f2a14f3a0c77f545de`. Rule list: `[]` (Enabled: `True`). {{(pid=80119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:36:43.683509 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-84e07d35-a106-4eff-b66a-1eda46682d3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:36:43.740927 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1144/2280] 10.222.0.49 () {58 vars in 957 bytes} [Wed Jun 15 10:36:43 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:36:44.041528 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4bd2bb-ffd0-467c-ae94-5d470aeb8b0c service nova] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:36:44.042994 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-8d4bd2bb-ffd0-467c-ae94-5d470aeb8b0c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:36:44.054693 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4bd2bb-ffd0-467c-ae94-5d470aeb8b0c service nova] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:36:44.058290 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96128cb-55c3-4687-b28f-f68df63f7c2a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0cb44a17d205431793e9a8791e477a32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '559b3e713f5844b6ae37151d56dd419f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:36:44.058923 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96128cb-55c3-4687-b28f-f68df63f7c2a service neutron] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:36:44.059240 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96128cb-55c3-4687-b28f-f68df63f7c2a service neutron] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:36:44.059521 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96128cb-55c3-4687-b28f-f68df63f7c2a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:36:44.061236 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-d96128cb-55c3-4687-b28f-f68df63f7c2a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:36:44.070847 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96128cb-55c3-4687-b28f-f68df63f7c2a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:36:44.072295 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96128cb-55c3-4687-b28f-f68df63f7c2a service neutron] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:36:44.080239 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1137/2281] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Jun 15 10:36:44 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:36:44.381432 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e07d35-a106-4eff-b66a-1eda46682d3d None None] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:36:44.382928 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-84e07d35-a106-4eff-b66a-1eda46682d3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:36:44.430956 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=100 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:36:44.455233 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e07d35-a106-4eff-b66a-1eda46682d3d None None] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:36:44.460249 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8e4a1a-7fa5-46bc-b93b-95b5b27245f4 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd03612a525644518517069b3fa43522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '559b3e713f5844b6ae37151d56dd419f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:36:44.460905 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8e4a1a-7fa5-46bc-b93b-95b5b27245f4 service placement] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:36:44.461194 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8e4a1a-7fa5-46bc-b93b-95b5b27245f4 service placement] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:36:44.461471 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8e4a1a-7fa5-46bc-b93b-95b5b27245f4 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:36:44.463403 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-0d8e4a1a-7fa5-46bc-b93b-95b5b27245f4 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:36:44.475012 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d8e4a1a-7fa5-46bc-b93b-95b5b27245f4 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:36:44.476630 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d8e4a1a-7fa5-46bc-b93b-95b5b27245f4 service placement] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:36:44.486972 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1145/2282] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Jun 15 10:36:44 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:36:47.285267 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96128cb-55c3-4687-b28f-f68df63f7c2a service neutron] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:36:47.293979 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96128cb-55c3-4687-b28f-f68df63f7c2a service neutron] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:36:47.298018 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5fc78b2-5f79-4b1f-984d-7e1b022e4746 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0cb44a17d205431793e9a8791e477a32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '559b3e713f5844b6ae37151d56dd419f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:36:47.298698 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fc78b2-5f79-4b1f-984d-7e1b022e4746 service neutron] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:36:47.299238 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fc78b2-5f79-4b1f-984d-7e1b022e4746 service neutron] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:36:47.299556 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fc78b2-5f79-4b1f-984d-7e1b022e4746 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:36:47.301367 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-a5fc78b2-5f79-4b1f-984d-7e1b022e4746 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:36:47.316845 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:36:47.382531 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5fc78b2-5f79-4b1f-984d-7e1b022e4746 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:36:47.384031 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5fc78b2-5f79-4b1f-984d-7e1b022e4746 service neutron] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:36:47.398539 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1138/2283] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Jun 15 10:36:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:36:50.737741 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8e4a1a-7fa5-46bc-b93b-95b5b27245f4 service placement] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:36:50.739230 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-0d8e4a1a-7fa5-46bc-b93b-95b5b27245f4 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:36:50.752815 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8e4a1a-7fa5-46bc-b93b-95b5b27245f4 service placement] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:36:50.757936 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a112484-3461-4177-87b7-bede99fb926b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd03612a525644518517069b3fa43522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '559b3e713f5844b6ae37151d56dd419f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:36:50.758585 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a112484-3461-4177-87b7-bede99fb926b service placement] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:36:50.758869 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a112484-3461-4177-87b7-bede99fb926b service placement] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:36:50.759183 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a112484-3461-4177-87b7-bede99fb926b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:36:50.760941 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-4a112484-3461-4177-87b7-bede99fb926b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:36:50.777470 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a112484-3461-4177-87b7-bede99fb926b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:36:50.779294 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a112484-3461-4177-87b7-bede99fb926b service placement] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:36:50.791963 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1146/2284] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Jun 15 10:36:50 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 10:36:50.793857 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5fc78b2-5f79-4b1f-984d-7e1b022e4746 service neutron] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:36:50.796316 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-a5fc78b2-5f79-4b1f-984d-7e1b022e4746 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:36:50.809986 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5fc78b2-5f79-4b1f-984d-7e1b022e4746 service neutron] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:36:50.815543 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b1c006-660b-4e28-82c6-a9820e18e696 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:36:50.817113 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b1c006-660b-4e28-82c6-a9820e18e696 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:36:50.817113 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b1c006-660b-4e28-82c6-a9820e18e696 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:36:50.817302 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b1c006-660b-4e28-82c6-a9820e18e696 admin admin] PATH_INFO: `/v3/users/6d152f74395c4b6cb889058f612d502f` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:36:50.818356 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b1c006-660b-4e28-82c6-a9820e18e696 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6d152f74395c4b6cb889058f612d502f)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:36:50.820319 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b1c006-660b-4e28-82c6-a9820e18e696 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:36:50.867220 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-91b1c006-660b-4e28-82c6-a9820e18e696 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6d152f74395c4b6cb889058f612d502f'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:36:50.873973 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-91b1c006-660b-4e28-82c6-a9820e18e696 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6d152f74395c4b6cb889058f612d502f'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:36:50.874634 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-91b1c006-660b-4e28-82c6-a9820e18e696 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6d152f74395c4b6cb889058f612d502f'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:36:50.875876 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-91b1c006-660b-4e28-82c6-a9820e18e696 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6d152f74395c4b6cb889058f612d502f'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:36:50.883055 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-91b1c006-660b-4e28-82c6-a9820e18e696 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6d152f74395c4b6cb889058f612d502f'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:36:50.899885 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1139/2285] 10.222.0.49 () {58 vars in 1223 bytes} [Wed Jun 15 10:36:50 2022] DELETE /identity/v3/users/6d152f74395c4b6cb889058f612d502f => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:36:51.103959 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a112484-3461-4177-87b7-bede99fb926b service placement] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:36:51.117548 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a112484-3461-4177-87b7-bede99fb926b service placement] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:36:51.178944 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2399bb-51d2-4f00-a3fb-49fed6f1d366 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:36:51.179859 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2399bb-51d2-4f00-a3fb-49fed6f1d366 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:36:51.180165 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2399bb-51d2-4f00-a3fb-49fed6f1d366 admin admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:36:51.180442 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2399bb-51d2-4f00-a3fb-49fed6f1d366 admin admin] PATH_INFO: `/v3/projects/993c57bae00b4a0db1d8de0f5017630a` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:36:51.181659 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f2399bb-51d2-4f00-a3fb-49fed6f1d366 admin admin] RBAC: Authorizing `identity:delete_project(project_id=993c57bae00b4a0db1d8de0f5017630a)` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:36:51.183657 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f2399bb-51d2-4f00-a3fb-49fed6f1d366 admin admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:36:51.231314 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-2f2399bb-51d2-4f00-a3fb-49fed6f1d366 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '993c57bae00b4a0db1d8de0f5017630a', 'request_id': 'req-2f2399bb-51d2-4f00-a3fb-49fed6f1d366'} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:36:51.239158 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-2f2399bb-51d2-4f00-a3fb-49fed6f1d366 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '993c57bae00b4a0db1d8de0f5017630a', 'request_id': 'req-2f2399bb-51d2-4f00-a3fb-49fed6f1d366'} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:36:51.244703 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-2f2399bb-51d2-4f00-a3fb-49fed6f1d366 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '993c57bae00b4a0db1d8de0f5017630a', 'request_id': 'req-2f2399bb-51d2-4f00-a3fb-49fed6f1d366'} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:36:51.255463 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-2f2399bb-51d2-4f00-a3fb-49fed6f1d366 admin admin] The token cache is being invalidate because project 993c57bae00b4a0db1d8de0f5017630a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80119) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 10:36:51.255751 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-2f2399bb-51d2-4f00-a3fb-49fed6f1d366 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:36:51.256375 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-2f2399bb-51d2-4f00-a3fb-49fed6f1d366 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:36:51.257903 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1147/2286] 10.222.0.49 () {58 vars in 1232 bytes} [Wed Jun 15 10:36:51 2022] DELETE /identity/v3/projects/993c57bae00b4a0db1d8de0f5017630a => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:36:51.262982 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0106075d-26d1-46dd-a1a4-41d4045fa9f4 None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:36:51.263305 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0106075d-26d1-46dd-a1a4-41d4045fa9f4 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:36:51.263585 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0106075d-26d1-46dd-a1a4-41d4045fa9f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:36:51.282107 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-0106075d-26d1-46dd-a1a4-41d4045fa9f4 None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:36:51.283349 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-0106075d-26d1-46dd-a1a4-41d4045fa9f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:36:51.340606 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1140/2287] 10.222.0.49 () {58 vars in 957 bytes} [Wed Jun 15 10:36:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:36:51.348938 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d810062-13dd-4212-b101-c2a119baf3e9 None None] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:36:51.349234 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d810062-13dd-4212-b101-c2a119baf3e9 None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:36:51.349543 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d810062-13dd-4212-b101-c2a119baf3e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:36:51.367384 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.auth.core [None req-5d810062-13dd-4212-b101-c2a119baf3e9 None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:36:51.368585 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-5d810062-13dd-4212-b101-c2a119baf3e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:36:51.376384 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1148/2288] 10.222.0.49 () {58 vars in 957 bytes} [Wed Jun 15 10:36:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:36:51.380543 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0106075d-26d1-46dd-a1a4-41d4045fa9f4 None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:36:51.390136 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0106075d-26d1-46dd-a1a4-41d4045fa9f4 None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:36:51.395312 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4d7773-891f-4d62-bbba-e9ee4fa87658 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:36:51.395965 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4d7773-891f-4d62-bbba-e9ee4fa87658 admin admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:36:51.396250 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4d7773-891f-4d62-bbba-e9ee4fa87658 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:36:51.396528 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4d7773-891f-4d62-bbba-e9ee4fa87658 admin admin] PATH_INFO: `/v3/domains` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:36:51.397255 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c4d7773-891f-4d62-bbba-e9ee4fa87658 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:36:51.397561 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c4d7773-891f-4d62-bbba-e9ee4fa87658 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:36:51.399179 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c4d7773-891f-4d62-bbba-e9ee4fa87658 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:36:51.411665 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1141/2289] 10.222.0.49 () {58 vars in 1152 bytes} [Wed Jun 15 10:36:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:36:51.415470 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d810062-13dd-4212-b101-c2a119baf3e9 None None] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:36:51.424552 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d810062-13dd-4212-b101-c2a119baf3e9 None None] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:36:51.429154 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c340eceb-bc5e-4854-9b26-89fd5c8a7f47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:36:51.429804 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c340eceb-bc5e-4854-9b26-89fd5c8a7f47 admin admin] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:36:51.430089 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c340eceb-bc5e-4854-9b26-89fd5c8a7f47 admin admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:36:51.430361 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c340eceb-bc5e-4854-9b26-89fd5c8a7f47 admin admin] PATH_INFO: `/v3/projects` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:36:51.431030 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c340eceb-bc5e-4854-9b26-89fd5c8a7f47 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:36:51.432947 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c340eceb-bc5e-4854-9b26-89fd5c8a7f47 admin admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:36:51.458382 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1149/2290] 10.222.0.49 () {60 vars in 1152 bytes} [Wed Jun 15 10:36:51 2022] POST /identity/v3/projects => generated 381 bytes in 43 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 10:36:51.462594 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4d7773-891f-4d62-bbba-e9ee4fa87658 admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:36:51.473213 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4d7773-891f-4d62-bbba-e9ee4fa87658 admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:36:51.478003 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:36:51.535023 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2133dbf1-5831-427f-931e-2ab523579c3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:36:51.535773 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2133dbf1-5831-427f-931e-2ab523579c3b admin admin] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:36:51.536065 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2133dbf1-5831-427f-931e-2ab523579c3b admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:36:51.536705 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2133dbf1-5831-427f-931e-2ab523579c3b admin admin] PATH_INFO: `/v3/users` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:36:51.537085 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2133dbf1-5831-427f-931e-2ab523579c3b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:36:51.539119 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2133dbf1-5831-427f-931e-2ab523579c3b admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:36:51.565017 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1142/2291] 10.222.0.49 () {60 vars in 1143 bytes} [Wed Jun 15 10:36:51 2022] POST /identity/v3/users => generated 291 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 10:36:51.566299 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c340eceb-bc5e-4854-9b26-89fd5c8a7f47 admin admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:36:51.577286 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c340eceb-bc5e-4854-9b26-89fd5c8a7f47 admin admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:36:51.584249 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9643167-5aff-42c7-abed-7491f53bb580 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:36:51.585228 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9643167-5aff-42c7-abed-7491f53bb580 admin admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:36:51.585578 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9643167-5aff-42c7-abed-7491f53bb580 admin admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:36:51.585955 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9643167-5aff-42c7-abed-7491f53bb580 admin admin] PATH_INFO: `/v3/roles` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:36:51.586814 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9643167-5aff-42c7-abed-7491f53bb580 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:36:51.589450 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9643167-5aff-42c7-abed-7491f53bb580 admin admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:36:51.607247 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1150/2292] 10.222.0.49 () {58 vars in 1121 bytes} [Wed Jun 15 10:36:51 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 10:36:51.608255 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2133dbf1-5831-427f-931e-2ab523579c3b admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:36:51.621879 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2133dbf1-5831-427f-931e-2ab523579c3b admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:36:51.628952 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45226f11-4177-408c-a0d8-9e0df052b242 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:36:51.630030 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45226f11-4177-408c-a0d8-9e0df052b242 admin admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:36:51.630147 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45226f11-4177-408c-a0d8-9e0df052b242 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:36:51.630534 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45226f11-4177-408c-a0d8-9e0df052b242 admin admin] PATH_INFO: `/v3/roles` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:36:51.631367 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45226f11-4177-408c-a0d8-9e0df052b242 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:36:51.633069 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45226f11-4177-408c-a0d8-9e0df052b242 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:36:51.645805 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1143/2293] 10.222.0.49 () {58 vars in 1121 bytes} [Wed Jun 15 10:36:51 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 10:36:51.650004 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9643167-5aff-42c7-abed-7491f53bb580 admin admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:36:51.660496 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9643167-5aff-42c7-abed-7491f53bb580 admin admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:36:51.665249 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3dc6208-7371-45a3-a6aa-d22441dc299b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:36:51.666144 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3dc6208-7371-45a3-a6aa-d22441dc299b admin admin] REQUEST_METHOD: `PUT` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:36:51.666428 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3dc6208-7371-45a3-a6aa-d22441dc299b admin admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:36:51.666712 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3dc6208-7371-45a3-a6aa-d22441dc299b admin admin] PATH_INFO: `/v3/projects/a61b0ca68bdd4d789a291981610da7c6/users/79f4b4fca1254c37b5639b2634f8db1f/roles/6876e57cc51c448097ab109d35b2a3da` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:36:51.671139 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:36:51.686616 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3dc6208-7371-45a3-a6aa-d22441dc299b admin admin] RBAC: Authorizing `identity:create_grant(project_id=a61b0ca68bdd4d789a291981610da7c6, user_id=79f4b4fca1254c37b5639b2634f8db1f, role_id=6876e57cc51c448097ab109d35b2a3da)` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:36:51.690301 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3dc6208-7371-45a3-a6aa-d22441dc299b admin admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:36:51.714606 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1151/2294] 10.222.0.49 () {60 vars in 1482 bytes} [Wed Jun 15 10:36:51 2022] PUT /identity/v3/projects/a61b0ca68bdd4d789a291981610da7c6/users/79f4b4fca1254c37b5639b2634f8db1f/roles/6876e57cc51c448097ab109d35b2a3da => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:36:51.724673 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1accc2f1-706c-4a5d-b43e-3015a02de707 None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:36:51.724972 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1accc2f1-706c-4a5d-b43e-3015a02de707 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:36:51.725261 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1accc2f1-706c-4a5d-b43e-3015a02de707 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:36:51.751653 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-1accc2f1-706c-4a5d-b43e-3015a02de707 None None] MFA Rules not processed for user `79f4b4fca1254c37b5639b2634f8db1f`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:36:51.752881 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-1accc2f1-706c-4a5d-b43e-3015a02de707 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:36:51.818393 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1144/2295] 10.222.0.49 () {58 vars in 957 bytes} [Wed Jun 15 10:36:51 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:36:51.822794 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3dc6208-7371-45a3-a6aa-d22441dc299b admin admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:36:51.884098 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3dc6208-7371-45a3-a6aa-d22441dc299b admin admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:36:51.888570 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b94ef3a-7d92-4902-a05d-437118b752b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:36:51.889219 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b94ef3a-7d92-4902-a05d-437118b752b8 admin admin] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:36:51.889502 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b94ef3a-7d92-4902-a05d-437118b752b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:36:51.889779 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b94ef3a-7d92-4902-a05d-437118b752b8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:36:51.890461 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b94ef3a-7d92-4902-a05d-437118b752b8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:36:51.892972 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b94ef3a-7d92-4902-a05d-437118b752b8 admin admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:36:51.920328 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1152/2296] 10.222.0.49 () {60 vars in 1152 bytes} [Wed Jun 15 10:36:51 2022] POST /identity/v3/projects => generated 381 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 10:36:51.921606 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1accc2f1-706c-4a5d-b43e-3015a02de707 None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:36:51.932160 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1accc2f1-706c-4a5d-b43e-3015a02de707 None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:36:51.986286 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e1da28-8f05-4a6d-8896-941d3b9813ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:36:51.986942 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e1da28-8f05-4a6d-8896-941d3b9813ea admin admin] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:36:51.987257 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e1da28-8f05-4a6d-8896-941d3b9813ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:36:51.987529 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e1da28-8f05-4a6d-8896-941d3b9813ea admin admin] PATH_INFO: `/v3/users` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:36:51.988210 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e1da28-8f05-4a6d-8896-941d3b9813ea admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:36:51.990107 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e1da28-8f05-4a6d-8896-941d3b9813ea admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:36:52.013331 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1145/2297] 10.222.0.49 () {60 vars in 1143 bytes} [Wed Jun 15 10:36:51 2022] POST /identity/v3/users => generated 297 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 10:36:52.014472 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b94ef3a-7d92-4902-a05d-437118b752b8 admin admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:36:52.025228 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b94ef3a-7d92-4902-a05d-437118b752b8 admin admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:36:52.031187 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d905ebe1-a786-4a26-a9bb-27f3116c5aa2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:36:52.031845 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d905ebe1-a786-4a26-a9bb-27f3116c5aa2 admin admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:36:52.032128 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d905ebe1-a786-4a26-a9bb-27f3116c5aa2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:36:52.032403 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d905ebe1-a786-4a26-a9bb-27f3116c5aa2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:36:52.033124 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d905ebe1-a786-4a26-a9bb-27f3116c5aa2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:36:52.034720 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d905ebe1-a786-4a26-a9bb-27f3116c5aa2 admin admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:36:52.045078 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1153/2298] 10.222.0.49 () {58 vars in 1121 bytes} [Wed Jun 15 10:36:52 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 10:36:52.049512 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e1da28-8f05-4a6d-8896-941d3b9813ea admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:36:52.059215 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e1da28-8f05-4a6d-8896-941d3b9813ea admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:36:52.064392 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c164f715-57c4-4561-9bd1-83048d7dcac0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:36:52.065256 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c164f715-57c4-4561-9bd1-83048d7dcac0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:36:52.065535 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c164f715-57c4-4561-9bd1-83048d7dcac0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:36:52.065805 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c164f715-57c4-4561-9bd1-83048d7dcac0 admin admin] PATH_INFO: `/v3/projects/d9557ccaa2164ef8affff21e40d5b2af/users/b86636923d2c404cb869a5154dbdeb1b/roles/8051174e7a7947d5b991ebd793c2e543` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:36:52.068007 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:36:52.081995 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c164f715-57c4-4561-9bd1-83048d7dcac0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d9557ccaa2164ef8affff21e40d5b2af, user_id=b86636923d2c404cb869a5154dbdeb1b, role_id=8051174e7a7947d5b991ebd793c2e543)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:36:52.084697 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c164f715-57c4-4561-9bd1-83048d7dcac0 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:36:52.099038 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1146/2299] 10.222.0.49 () {60 vars in 1482 bytes} [Wed Jun 15 10:36:52 2022] PUT /identity/v3/projects/d9557ccaa2164ef8affff21e40d5b2af/users/b86636923d2c404cb869a5154dbdeb1b/roles/8051174e7a7947d5b991ebd793c2e543 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 10:36:52.103012 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d905ebe1-a786-4a26-a9bb-27f3116c5aa2 admin admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:36:52.113958 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d905ebe1-a786-4a26-a9bb-27f3116c5aa2 admin admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:36:52.170870 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12cecc79-5453-4959-b918-8040e1515609 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:36:52.171509 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cecc79-5453-4959-b918-8040e1515609 admin admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:36:52.172290 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cecc79-5453-4959-b918-8040e1515609 admin admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:36:52.172573 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cecc79-5453-4959-b918-8040e1515609 admin admin] PATH_INFO: `/v3/roles` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:36:52.173299 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12cecc79-5453-4959-b918-8040e1515609 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:36:52.174906 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12cecc79-5453-4959-b918-8040e1515609 admin admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:36:52.185435 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1154/2300] 10.222.0.49 () {58 vars in 1121 bytes} [Wed Jun 15 10:36:52 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 10:36:52.189810 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c164f715-57c4-4561-9bd1-83048d7dcac0 admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:36:52.200525 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c164f715-57c4-4561-9bd1-83048d7dcac0 admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:36:52.208624 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f5f020-0294-4a90-9e36-df57fd1396c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:36:52.209652 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f5f020-0294-4a90-9e36-df57fd1396c0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:36:52.209938 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f5f020-0294-4a90-9e36-df57fd1396c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:36:52.210215 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f5f020-0294-4a90-9e36-df57fd1396c0 admin admin] PATH_INFO: `/v3/projects/d9557ccaa2164ef8affff21e40d5b2af/users/b86636923d2c404cb869a5154dbdeb1b/roles/8051174e7a7947d5b991ebd793c2e543` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:36:52.212252 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f5f020-0294-4a90-9e36-df57fd1396c0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d9557ccaa2164ef8affff21e40d5b2af, user_id=b86636923d2c404cb869a5154dbdeb1b, role_id=8051174e7a7947d5b991ebd793c2e543)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:36:52.215593 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f5f020-0294-4a90-9e36-df57fd1396c0 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:36:52.232433 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1147/2301] 10.222.0.49 () {60 vars in 1482 bytes} [Wed Jun 15 10:36:52 2022] PUT /identity/v3/projects/d9557ccaa2164ef8affff21e40d5b2af/users/b86636923d2c404cb869a5154dbdeb1b/roles/8051174e7a7947d5b991ebd793c2e543 => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:36:52.244107 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda4256a-7cb6-4861-b103-35080d163cb8 None None] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:36:52.244376 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda4256a-7cb6-4861-b103-35080d163cb8 None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:36:52.244696 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda4256a-7cb6-4861-b103-35080d163cb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:36:52.270656 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.auth.core [None req-eda4256a-7cb6-4861-b103-35080d163cb8 None None] MFA Rules not processed for user `b86636923d2c404cb869a5154dbdeb1b`. Rule list: `[]` (Enabled: `True`). {{(pid=80119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:36:52.271920 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-eda4256a-7cb6-4861-b103-35080d163cb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:36:52.315302 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:36:52.340732 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1155/2302] 10.222.0.49 () {58 vars in 957 bytes} [Wed Jun 15 10:36:52 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:36:52.352003 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f5f020-0294-4a90-9e36-df57fd1396c0 admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:36:52.353724 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-c3f5f020-0294-4a90-9e36-df57fd1396c0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:36:52.411026 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f5f020-0294-4a90-9e36-df57fd1396c0 admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:36:52.414676 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2511690-7007-445f-844e-e91093c32c04 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20d1327553404b80aecee87c78fcb52b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '559b3e713f5844b6ae37151d56dd419f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:36:52.415372 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2511690-7007-445f-844e-e91093c32c04 service cinder] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:36:52.415659 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2511690-7007-445f-844e-e91093c32c04 service cinder] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:36:52.415936 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2511690-7007-445f-844e-e91093c32c04 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:36:52.425370 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2511690-7007-445f-844e-e91093c32c04 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:36:52.426732 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2511690-7007-445f-844e-e91093c32c04 service cinder] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:36:52.436039 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1148/2303] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Jun 15 10:36:52 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:36:52.506655 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda4256a-7cb6-4861-b103-35080d163cb8 None None] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:36:52.516485 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda4256a-7cb6-4861-b103-35080d163cb8 None None] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:36:52.520127 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9375c69c-8b73-4fdd-b72e-a53a0ff1955e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20d1327553404b80aecee87c78fcb52b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '559b3e713f5844b6ae37151d56dd419f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:36:52.520771 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9375c69c-8b73-4fdd-b72e-a53a0ff1955e service cinder] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:36:52.521052 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9375c69c-8b73-4fdd-b72e-a53a0ff1955e service cinder] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:36:52.521355 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9375c69c-8b73-4fdd-b72e-a53a0ff1955e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:36:52.576483 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9375c69c-8b73-4fdd-b72e-a53a0ff1955e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:36:52.577911 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9375c69c-8b73-4fdd-b72e-a53a0ff1955e service cinder] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:36:52.586676 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1156/2304] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Jun 15 10:36:52 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:36:53.071294 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2511690-7007-445f-844e-e91093c32c04 service cinder] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:36:53.072882 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-f2511690-7007-445f-844e-e91093c32c04 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:36:53.151188 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2511690-7007-445f-844e-e91093c32c04 service cinder] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:36:53.157064 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19678b25-13f9-4ea0-b9e1-a2df14ff1807 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ac54528370d4747b6e8777fba9d0c82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '559b3e713f5844b6ae37151d56dd419f', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:36:53.157714 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19678b25-13f9-4ea0-b9e1-a2df14ff1807 service nova] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:36:53.158000 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19678b25-13f9-4ea0-b9e1-a2df14ff1807 service nova] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:36:53.158280 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19678b25-13f9-4ea0-b9e1-a2df14ff1807 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:36:53.160089 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-19678b25-13f9-4ea0-b9e1-a2df14ff1807 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:36:53.171871 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19678b25-13f9-4ea0-b9e1-a2df14ff1807 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:36:53.173486 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19678b25-13f9-4ea0-b9e1-a2df14ff1807 service nova] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:36:53.183873 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1149/2305] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Jun 15 10:36:53 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:36:59.421479 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb30b88-1d8f-4823-8a88-0746cb7a8569 None None] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:36:59.422010 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb30b88-1d8f-4823-8a88-0746cb7a8569 None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:36:59.422109 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb30b88-1d8f-4823-8a88-0746cb7a8569 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:36:59.441559 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.auth.core [None req-9bb30b88-1d8f-4823-8a88-0746cb7a8569 None None] MFA Rules not processed for user `103e10244eca444b86b9c51ff314d445`. Rule list: `[]` (Enabled: `True`). {{(pid=80119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:36:59.443074 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-9bb30b88-1d8f-4823-8a88-0746cb7a8569 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:36:59.504546 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1157/2306] 10.222.0.49 () {58 vars in 1009 bytes} [Wed Jun 15 10:36:59 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:36:59.517336 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19678b25-13f9-4ea0-b9e1-a2df14ff1807 service nova] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:36:59.519593 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-19678b25-13f9-4ea0-b9e1-a2df14ff1807 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:36:59.597925 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:36:59.630245 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19678b25-13f9-4ea0-b9e1-a2df14ff1807 service nova] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:36:59.636347 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec21464-4aed-4561-900d-83c32d390a69 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a69f99210444f9696e8b22615fb4a41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '559b3e713f5844b6ae37151d56dd419f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:36:59.637055 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec21464-4aed-4561-900d-83c32d390a69 service swift] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:36:59.637345 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec21464-4aed-4561-900d-83c32d390a69 service swift] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:36:59.637625 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec21464-4aed-4561-900d-83c32d390a69 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:36:59.647311 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec21464-4aed-4561-900d-83c32d390a69 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:36:59.648964 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec21464-4aed-4561-900d-83c32d390a69 service swift] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:36:59.658862 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1150/2307] 10.222.0.49 () {60 vars in 1384 bytes} [Wed Jun 15 10:36:59 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 15 10:37:02.596190 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb30b88-1d8f-4823-8a88-0746cb7a8569 None None] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:37:02.675608 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb30b88-1d8f-4823-8a88-0746cb7a8569 None None] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:37:02.680252 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6e21e3-cbc8-4a76-a634-f1a50603def2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:37:02.681203 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6e21e3-cbc8-4a76-a634-f1a50603def2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:37:02.681591 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6e21e3-cbc8-4a76-a634-f1a50603def2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:37:02.681910 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6e21e3-cbc8-4a76-a634-f1a50603def2 admin admin] PATH_INFO: `/v3/users/79f4b4fca1254c37b5639b2634f8db1f` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:37:02.686570 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c6e21e3-cbc8-4a76-a634-f1a50603def2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=79f4b4fca1254c37b5639b2634f8db1f)` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:37:02.688319 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c6e21e3-cbc8-4a76-a634-f1a50603def2 admin admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:37:02.723341 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:37:02.750180 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-3c6e21e3-cbc8-4a76-a634-f1a50603def2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '79f4b4fca1254c37b5639b2634f8db1f'} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:37:02.757430 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-3c6e21e3-cbc8-4a76-a634-f1a50603def2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '79f4b4fca1254c37b5639b2634f8db1f'} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:37:02.758192 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-3c6e21e3-cbc8-4a76-a634-f1a50603def2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '79f4b4fca1254c37b5639b2634f8db1f'} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:37:02.761694 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-3c6e21e3-cbc8-4a76-a634-f1a50603def2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '79f4b4fca1254c37b5639b2634f8db1f'} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:37:02.773342 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-3c6e21e3-cbc8-4a76-a634-f1a50603def2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '79f4b4fca1254c37b5639b2634f8db1f'} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:37:02.800302 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1158/2308] 10.222.0.49 () {58 vars in 1223 bytes} [Wed Jun 15 10:37:02 2022] DELETE /identity/v3/users/79f4b4fca1254c37b5639b2634f8db1f => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:37:02.805003 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1342e4-3da2-45bb-bb76-83e5e828282f None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:37:02.805600 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1342e4-3da2-45bb-bb76-83e5e828282f None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:37:02.805600 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1342e4-3da2-45bb-bb76-83e5e828282f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:37:02.824182 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-be1342e4-3da2-45bb-bb76-83e5e828282f None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:37:02.825698 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-be1342e4-3da2-45bb-bb76-83e5e828282f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:37:02.892476 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1151/2309] 10.222.0.49 () {58 vars in 957 bytes} [Wed Jun 15 10:37:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:37:02.903078 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6e21e3-cbc8-4a76-a634-f1a50603def2 admin admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:37:02.904625 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-3c6e21e3-cbc8-4a76-a634-f1a50603def2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:37:02.968302 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6e21e3-cbc8-4a76-a634-f1a50603def2 admin admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:37:02.973158 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02142dc3-a481-4102-8258-37116563c6bd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0cb44a17d205431793e9a8791e477a32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '559b3e713f5844b6ae37151d56dd419f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:37:02.974229 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02142dc3-a481-4102-8258-37116563c6bd service neutron] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:37:02.974229 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02142dc3-a481-4102-8258-37116563c6bd service neutron] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:37:02.974570 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02142dc3-a481-4102-8258-37116563c6bd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:37:02.985982 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02142dc3-a481-4102-8258-37116563c6bd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:37:02.987642 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02142dc3-a481-4102-8258-37116563c6bd service neutron] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:37:03.000991 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1159/2310] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Jun 15 10:37:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:37:03.380027 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1342e4-3da2-45bb-bb76-83e5e828282f None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:37:03.381483 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-be1342e4-3da2-45bb-bb76-83e5e828282f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:37:03.396864 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1342e4-3da2-45bb-bb76-83e5e828282f None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:37:03.401832 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7af7dc-986d-4fb8-be74-a5072a956f31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:37:03.402704 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7af7dc-986d-4fb8-be74-a5072a956f31 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:37:03.402996 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7af7dc-986d-4fb8-be74-a5072a956f31 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:37:03.403308 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7af7dc-986d-4fb8-be74-a5072a956f31 admin admin] PATH_INFO: `/v3/projects/a61b0ca68bdd4d789a291981610da7c6` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:37:03.404506 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7af7dc-986d-4fb8-be74-a5072a956f31 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a61b0ca68bdd4d789a291981610da7c6)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:37:03.406466 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7af7dc-986d-4fb8-be74-a5072a956f31 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:37:03.471816 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-ce7af7dc-986d-4fb8-be74-a5072a956f31 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a61b0ca68bdd4d789a291981610da7c6', 'request_id': 'req-ce7af7dc-986d-4fb8-be74-a5072a956f31'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:37:03.484354 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-ce7af7dc-986d-4fb8-be74-a5072a956f31 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a61b0ca68bdd4d789a291981610da7c6', 'request_id': 'req-ce7af7dc-986d-4fb8-be74-a5072a956f31'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:37:03.491741 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-ce7af7dc-986d-4fb8-be74-a5072a956f31 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a61b0ca68bdd4d789a291981610da7c6', 'request_id': 'req-ce7af7dc-986d-4fb8-be74-a5072a956f31'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:37:03.505468 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-ce7af7dc-986d-4fb8-be74-a5072a956f31 admin admin] The token cache is being invalidate because project a61b0ca68bdd4d789a291981610da7c6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80120) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 10:37:03.505756 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-ce7af7dc-986d-4fb8-be74-a5072a956f31 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:37:03.506417 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-ce7af7dc-986d-4fb8-be74-a5072a956f31 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:37:03.508292 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1152/2311] 10.222.0.49 () {58 vars in 1232 bytes} [Wed Jun 15 10:37:03 2022] DELETE /identity/v3/projects/a61b0ca68bdd4d789a291981610da7c6 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:37:03.512498 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02142dc3-a481-4102-8258-37116563c6bd service neutron] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:37:03.527198 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02142dc3-a481-4102-8258-37116563c6bd service neutron] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:37:03.597983 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2a4e0d-eb53-4c6d-82fd-99c1895b5342 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:37:03.598887 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2a4e0d-eb53-4c6d-82fd-99c1895b5342 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:37:03.599195 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2a4e0d-eb53-4c6d-82fd-99c1895b5342 admin admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:37:03.599473 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2a4e0d-eb53-4c6d-82fd-99c1895b5342 admin admin] PATH_INFO: `/v3/users/b86636923d2c404cb869a5154dbdeb1b` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:37:03.602247 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c2a4e0d-eb53-4c6d-82fd-99c1895b5342 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b86636923d2c404cb869a5154dbdeb1b)` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:37:03.602827 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c2a4e0d-eb53-4c6d-82fd-99c1895b5342 admin admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:37:03.666057 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-0c2a4e0d-eb53-4c6d-82fd-99c1895b5342 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b86636923d2c404cb869a5154dbdeb1b'} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:37:03.667871 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:DELETE=13,keystone:SELECT=86,keystone:INSERT=1 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:37:03.677440 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-0c2a4e0d-eb53-4c6d-82fd-99c1895b5342 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b86636923d2c404cb869a5154dbdeb1b'} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:37:03.678988 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-0c2a4e0d-eb53-4c6d-82fd-99c1895b5342 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b86636923d2c404cb869a5154dbdeb1b'} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:37:03.680365 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-0c2a4e0d-eb53-4c6d-82fd-99c1895b5342 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b86636923d2c404cb869a5154dbdeb1b'} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:37:03.690556 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-0c2a4e0d-eb53-4c6d-82fd-99c1895b5342 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b86636923d2c404cb869a5154dbdeb1b'} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:37:03.713066 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1160/2312] 10.222.0.49 () {58 vars in 1223 bytes} [Wed Jun 15 10:37:03 2022] DELETE /identity/v3/users/b86636923d2c404cb869a5154dbdeb1b => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:37:04.034096 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7af7dc-986d-4fb8-be74-a5072a956f31 admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:37:04.035794 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-ce7af7dc-986d-4fb8-be74-a5072a956f31 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:37:04.052840 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7af7dc-986d-4fb8-be74-a5072a956f31 admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:37:04.121758 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ab9777-d694-473b-86aa-cf719e2f6b67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:37:04.122878 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ab9777-d694-473b-86aa-cf719e2f6b67 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:37:04.123264 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ab9777-d694-473b-86aa-cf719e2f6b67 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:37:04.123617 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ab9777-d694-473b-86aa-cf719e2f6b67 admin admin] PATH_INFO: `/v3/projects/d9557ccaa2164ef8affff21e40d5b2af` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:37:04.125034 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ab9777-d694-473b-86aa-cf719e2f6b67 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d9557ccaa2164ef8affff21e40d5b2af)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:37:04.127277 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ab9777-d694-473b-86aa-cf719e2f6b67 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:37:04.129220 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:37:04.183916 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-d3ab9777-d694-473b-86aa-cf719e2f6b67 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd9557ccaa2164ef8affff21e40d5b2af', 'request_id': 'req-d3ab9777-d694-473b-86aa-cf719e2f6b67'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:37:04.197975 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-d3ab9777-d694-473b-86aa-cf719e2f6b67 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd9557ccaa2164ef8affff21e40d5b2af', 'request_id': 'req-d3ab9777-d694-473b-86aa-cf719e2f6b67'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:37:04.204336 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-d3ab9777-d694-473b-86aa-cf719e2f6b67 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd9557ccaa2164ef8affff21e40d5b2af', 'request_id': 'req-d3ab9777-d694-473b-86aa-cf719e2f6b67'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:37:04.216716 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-d3ab9777-d694-473b-86aa-cf719e2f6b67 admin admin] The token cache is being invalidate because project d9557ccaa2164ef8affff21e40d5b2af was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80120) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 10:37:04.217015 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-d3ab9777-d694-473b-86aa-cf719e2f6b67 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:37:04.217821 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-d3ab9777-d694-473b-86aa-cf719e2f6b67 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:37:04.220469 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1153/2313] 10.222.0.49 () {58 vars in 1232 bytes} [Wed Jun 15 10:37:04 2022] DELETE /identity/v3/projects/d9557ccaa2164ef8affff21e40d5b2af => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:37:04.225070 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d15801-4e57-49d8-a955-6f553bf3bf44 None None] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:37:04.225369 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d15801-4e57-49d8-a955-6f553bf3bf44 None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:37:04.225655 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d15801-4e57-49d8-a955-6f553bf3bf44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:37:04.249017 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.auth.core [None req-47d15801-4e57-49d8-a955-6f553bf3bf44 None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:37:04.250553 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-47d15801-4e57-49d8-a955-6f553bf3bf44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:37:04.319849 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1161/2314] 10.222.0.49 () {58 vars in 957 bytes} [Wed Jun 15 10:37:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 10:37:04.327970 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ae69a0-7a6e-485b-b931-742c2373f399 None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:37:04.328269 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ae69a0-7a6e-485b-b931-742c2373f399 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:37:04.328556 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ae69a0-7a6e-485b-b931-742c2373f399 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:37:04.353618 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-46ae69a0-7a6e-485b-b931-742c2373f399 None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:37:04.355563 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-46ae69a0-7a6e-485b-b931-742c2373f399 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:37:04.367138 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1154/2315] 10.222.0.49 () {58 vars in 957 bytes} [Wed Jun 15 10:37:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:37:04.369166 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d15801-4e57-49d8-a955-6f553bf3bf44 None None] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:37:04.389503 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d15801-4e57-49d8-a955-6f553bf3bf44 None None] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:37:04.394643 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42562f0-4562-410b-9120-3458f11675c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:37:04.395364 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42562f0-4562-410b-9120-3458f11675c6 admin admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:37:04.395655 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42562f0-4562-410b-9120-3458f11675c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:37:04.396083 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42562f0-4562-410b-9120-3458f11675c6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:37:04.396833 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c42562f0-4562-410b-9120-3458f11675c6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:37:04.397143 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c42562f0-4562-410b-9120-3458f11675c6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:37:04.398771 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c42562f0-4562-410b-9120-3458f11675c6 admin admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:37:04.416208 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1162/2316] 10.222.0.49 () {58 vars in 1152 bytes} [Wed Jun 15 10:37:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:37:04.417359 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ae69a0-7a6e-485b-b931-742c2373f399 None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:37:04.427658 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ae69a0-7a6e-485b-b931-742c2373f399 None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:37:04.432642 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1207dd-c879-476e-b743-b32c53a7cc17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:37:04.433323 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1207dd-c879-476e-b743-b32c53a7cc17 admin admin] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:37:04.433615 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1207dd-c879-476e-b743-b32c53a7cc17 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:37:04.433896 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1207dd-c879-476e-b743-b32c53a7cc17 admin admin] PATH_INFO: `/v3/projects` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:37:04.434599 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1207dd-c879-476e-b743-b32c53a7cc17 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:37:04.436555 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1207dd-c879-476e-b743-b32c53a7cc17 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:37:04.467149 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1155/2317] 10.222.0.49 () {60 vars in 1152 bytes} [Wed Jun 15 10:37:04 2022] POST /identity/v3/projects => generated 363 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 10:37:04.471772 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42562f0-4562-410b-9120-3458f11675c6 admin admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:37:04.482145 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42562f0-4562-410b-9120-3458f11675c6 admin admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:37:04.565785 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded185f0-95df-4efb-ad21-bd6665a29ae9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:37:04.566470 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded185f0-95df-4efb-ad21-bd6665a29ae9 admin admin] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:37:04.566751 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded185f0-95df-4efb-ad21-bd6665a29ae9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:37:04.567030 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded185f0-95df-4efb-ad21-bd6665a29ae9 admin admin] PATH_INFO: `/v3/users` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:37:04.567769 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded185f0-95df-4efb-ad21-bd6665a29ae9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:37:04.569684 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded185f0-95df-4efb-ad21-bd6665a29ae9 admin admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:37:04.597816 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1163/2318] 10.222.0.49 () {60 vars in 1143 bytes} [Wed Jun 15 10:37:04 2022] POST /identity/v3/users => generated 282 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 10:37:04.601983 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1207dd-c879-476e-b743-b32c53a7cc17 admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:37:04.613145 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1207dd-c879-476e-b743-b32c53a7cc17 admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:37:04.618237 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ed7cb4-b232-4e6f-bacc-c8117d27afc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:37:04.618911 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ed7cb4-b232-4e6f-bacc-c8117d27afc9 admin admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:37:04.619230 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ed7cb4-b232-4e6f-bacc-c8117d27afc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:37:04.619513 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ed7cb4-b232-4e6f-bacc-c8117d27afc9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:37:04.620289 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ed7cb4-b232-4e6f-bacc-c8117d27afc9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:37:04.622638 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ed7cb4-b232-4e6f-bacc-c8117d27afc9 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:37:04.639374 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1156/2319] 10.222.0.49 () {58 vars in 1121 bytes} [Wed Jun 15 10:37:04 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 10:37:04.641342 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded185f0-95df-4efb-ad21-bd6665a29ae9 admin admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:37:04.655450 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded185f0-95df-4efb-ad21-bd6665a29ae9 admin admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:37:04.662051 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8df40ef-18f5-4d29-ab2e-ee8a4e0a6491 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:37:04.662787 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8df40ef-18f5-4d29-ab2e-ee8a4e0a6491 admin admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:37:04.663292 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8df40ef-18f5-4d29-ab2e-ee8a4e0a6491 admin admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:37:04.663512 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8df40ef-18f5-4d29-ab2e-ee8a4e0a6491 admin admin] PATH_INFO: `/v3/roles` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:37:04.664670 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8df40ef-18f5-4d29-ab2e-ee8a4e0a6491 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:37:04.666432 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8df40ef-18f5-4d29-ab2e-ee8a4e0a6491 admin admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:37:04.681346 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1164/2320] 10.222.0.49 () {58 vars in 1121 bytes} [Wed Jun 15 10:37:04 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 15 10:37:04.685477 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ed7cb4-b232-4e6f-bacc-c8117d27afc9 admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:37:04.698464 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ed7cb4-b232-4e6f-bacc-c8117d27afc9 admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:37:04.704283 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830f56e6-83d5-422d-8fc7-1fa44ff651bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:37:04.705479 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830f56e6-83d5-422d-8fc7-1fa44ff651bd admin admin] REQUEST_METHOD: `PUT` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:37:04.705828 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830f56e6-83d5-422d-8fc7-1fa44ff651bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:37:04.706168 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830f56e6-83d5-422d-8fc7-1fa44ff651bd admin admin] PATH_INFO: `/v3/projects/b276120ccf5945a4983d2e98d3234bc5/users/73f5d8a1daab4cf0afde2a9b1c5616e5/roles/6876e57cc51c448097ab109d35b2a3da` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:37:04.726526 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-830f56e6-83d5-422d-8fc7-1fa44ff651bd admin admin] RBAC: Authorizing `identity:create_grant(project_id=b276120ccf5945a4983d2e98d3234bc5, user_id=73f5d8a1daab4cf0afde2a9b1c5616e5, role_id=6876e57cc51c448097ab109d35b2a3da)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:37:04.731914 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-830f56e6-83d5-422d-8fc7-1fa44ff651bd admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:37:04.756157 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1157/2321] 10.222.0.49 () {60 vars in 1482 bytes} [Wed Jun 15 10:37:04 2022] PUT /identity/v3/projects/b276120ccf5945a4983d2e98d3234bc5/users/73f5d8a1daab4cf0afde2a9b1c5616e5/roles/6876e57cc51c448097ab109d35b2a3da => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:37:04.757907 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfcf4ec-f3a2-4b1c-8511-82ef3a6491a8 None None] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:37:04.758205 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfcf4ec-f3a2-4b1c-8511-82ef3a6491a8 None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:37:04.758580 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfcf4ec-f3a2-4b1c-8511-82ef3a6491a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:37:04.799884 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.auth.core [None req-9bfcf4ec-f3a2-4b1c-8511-82ef3a6491a8 None None] MFA Rules not processed for user `73f5d8a1daab4cf0afde2a9b1c5616e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:37:04.799884 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-9bfcf4ec-f3a2-4b1c-8511-82ef3a6491a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:37:04.821850 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=93,keystone:INSERT=4,keystone:DELETE=3 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:37:04.884484 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1165/2322] 10.222.0.49 () {58 vars in 957 bytes} [Wed Jun 15 10:37:04 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:37:04.891685 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830f56e6-83d5-422d-8fc7-1fa44ff651bd admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:37:04.963310 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830f56e6-83d5-422d-8fc7-1fa44ff651bd admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:37:04.968301 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445c8c71-b00d-4feb-97c9-f06c651a5891 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:37:04.968968 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445c8c71-b00d-4feb-97c9-f06c651a5891 admin admin] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:37:04.969255 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445c8c71-b00d-4feb-97c9-f06c651a5891 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:37:04.969531 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445c8c71-b00d-4feb-97c9-f06c651a5891 admin admin] PATH_INFO: `/v3/projects` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:37:04.970231 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-445c8c71-b00d-4feb-97c9-f06c651a5891 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:37:04.972282 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-445c8c71-b00d-4feb-97c9-f06c651a5891 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:37:05.004281 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1158/2323] 10.222.0.49 () {60 vars in 1152 bytes} [Wed Jun 15 10:37:04 2022] POST /identity/v3/projects => generated 365 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 10:37:05.008020 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bfcf4ec-f3a2-4b1c-8511-82ef3a6491a8 None None] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:37:05.020697 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bfcf4ec-f3a2-4b1c-8511-82ef3a6491a8 None None] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:37:05.089809 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806e7566-271d-4769-bc0a-897b7c4945eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:37:05.090581 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806e7566-271d-4769-bc0a-897b7c4945eb admin admin] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:37:05.090935 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806e7566-271d-4769-bc0a-897b7c4945eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:37:05.091310 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806e7566-271d-4769-bc0a-897b7c4945eb admin admin] PATH_INFO: `/v3/users` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:37:05.092115 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-806e7566-271d-4769-bc0a-897b7c4945eb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:37:05.094253 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-806e7566-271d-4769-bc0a-897b7c4945eb admin admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:37:05.119124 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1166/2324] 10.222.0.49 () {60 vars in 1143 bytes} [Wed Jun 15 10:37:05 2022] POST /identity/v3/users => generated 289 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 10:37:05.122340 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445c8c71-b00d-4feb-97c9-f06c651a5891 admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:37:05.134175 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445c8c71-b00d-4feb-97c9-f06c651a5891 admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:37:05.139377 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aefe8c44-fd16-4774-ac86-ff2d9f046541 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:37:05.140787 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefe8c44-fd16-4774-ac86-ff2d9f046541 admin admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:37:05.141134 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefe8c44-fd16-4774-ac86-ff2d9f046541 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:37:05.141478 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefe8c44-fd16-4774-ac86-ff2d9f046541 admin admin] PATH_INFO: `/v3/roles` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:37:05.142282 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aefe8c44-fd16-4774-ac86-ff2d9f046541 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:37:05.144047 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aefe8c44-fd16-4774-ac86-ff2d9f046541 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:37:05.150047 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:37:05.157261 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1159/2325] 10.222.0.49 () {58 vars in 1121 bytes} [Wed Jun 15 10:37:05 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 10:37:05.161675 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806e7566-271d-4769-bc0a-897b7c4945eb admin admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:37:05.172846 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806e7566-271d-4769-bc0a-897b7c4945eb admin admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:37:05.177728 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25cd30a-43ff-46fb-b701-0072a0b36863 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:37:05.179024 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25cd30a-43ff-46fb-b701-0072a0b36863 admin admin] REQUEST_METHOD: `PUT` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:37:05.179345 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25cd30a-43ff-46fb-b701-0072a0b36863 admin admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:37:05.179621 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25cd30a-43ff-46fb-b701-0072a0b36863 admin admin] PATH_INFO: `/v3/projects/bd5cba6852ed4e0f92c00b29b92b8764/users/95f27e5f4a204fe48bea0fa1f7b89903/roles/8051174e7a7947d5b991ebd793c2e543` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:37:05.197721 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e25cd30a-43ff-46fb-b701-0072a0b36863 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bd5cba6852ed4e0f92c00b29b92b8764, user_id=95f27e5f4a204fe48bea0fa1f7b89903, role_id=8051174e7a7947d5b991ebd793c2e543)` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:37:05.200539 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e25cd30a-43ff-46fb-b701-0072a0b36863 admin admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:37:05.217171 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1167/2326] 10.222.0.49 () {60 vars in 1482 bytes} [Wed Jun 15 10:37:05 2022] PUT /identity/v3/projects/bd5cba6852ed4e0f92c00b29b92b8764/users/95f27e5f4a204fe48bea0fa1f7b89903/roles/8051174e7a7947d5b991ebd793c2e543 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:37:05.223233 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aefe8c44-fd16-4774-ac86-ff2d9f046541 admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:37:05.240053 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aefe8c44-fd16-4774-ac86-ff2d9f046541 admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:37:05.300620 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875c2fe4-3eb4-470b-a88d-cd269c267d10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:37:05.301305 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875c2fe4-3eb4-470b-a88d-cd269c267d10 admin admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:37:05.301589 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875c2fe4-3eb4-470b-a88d-cd269c267d10 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:37:05.301862 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875c2fe4-3eb4-470b-a88d-cd269c267d10 admin admin] PATH_INFO: `/v3/roles` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:37:05.302590 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-875c2fe4-3eb4-470b-a88d-cd269c267d10 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:37:05.304245 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-875c2fe4-3eb4-470b-a88d-cd269c267d10 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:37:05.315619 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1160/2327] 10.222.0.49 () {58 vars in 1121 bytes} [Wed Jun 15 10:37:05 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 15 10:37:05.320176 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25cd30a-43ff-46fb-b701-0072a0b36863 admin admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:37:05.331222 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25cd30a-43ff-46fb-b701-0072a0b36863 admin admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:37:05.336265 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2579aa-ac00-44b2-8990-4c3d8f723580 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:37:05.337403 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2579aa-ac00-44b2-8990-4c3d8f723580 admin admin] REQUEST_METHOD: `PUT` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:37:05.337777 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2579aa-ac00-44b2-8990-4c3d8f723580 admin admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:37:05.338126 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2579aa-ac00-44b2-8990-4c3d8f723580 admin admin] PATH_INFO: `/v3/projects/bd5cba6852ed4e0f92c00b29b92b8764/users/95f27e5f4a204fe48bea0fa1f7b89903/roles/8051174e7a7947d5b991ebd793c2e543` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:37:05.340328 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2579aa-ac00-44b2-8990-4c3d8f723580 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bd5cba6852ed4e0f92c00b29b92b8764, user_id=95f27e5f4a204fe48bea0fa1f7b89903, role_id=8051174e7a7947d5b991ebd793c2e543)` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:37:05.343205 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2579aa-ac00-44b2-8990-4c3d8f723580 admin admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:37:05.354650 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1168/2328] 10.222.0.49 () {60 vars in 1482 bytes} [Wed Jun 15 10:37:05 2022] PUT /identity/v3/projects/bd5cba6852ed4e0f92c00b29b92b8764/users/95f27e5f4a204fe48bea0fa1f7b89903/roles/8051174e7a7947d5b991ebd793c2e543 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 10:37:05.366360 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfd0b90-ee79-43d8-a173-f87eca7622af None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:37:05.366775 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfd0b90-ee79-43d8-a173-f87eca7622af None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:37:05.367059 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfd0b90-ee79-43d8-a173-f87eca7622af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:37:05.417348 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-5dfd0b90-ee79-43d8-a173-f87eca7622af None None] MFA Rules not processed for user `95f27e5f4a204fe48bea0fa1f7b89903`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:37:05.418599 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-5dfd0b90-ee79-43d8-a173-f87eca7622af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:37:05.508487 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1161/2329] 10.222.0.49 () {58 vars in 957 bytes} [Wed Jun 15 10:37:05 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:37:05.516408 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2579aa-ac00-44b2-8990-4c3d8f723580 admin admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:37:05.518098 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-6d2579aa-ac00-44b2-8990-4c3d8f723580 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:37:05.594752 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2579aa-ac00-44b2-8990-4c3d8f723580 admin admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:37:05.602346 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9aa436a-6517-478c-9e96-eaa7bf6d879f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20d1327553404b80aecee87c78fcb52b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '559b3e713f5844b6ae37151d56dd419f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:37:05.603011 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9aa436a-6517-478c-9e96-eaa7bf6d879f service cinder] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:37:05.603336 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9aa436a-6517-478c-9e96-eaa7bf6d879f service cinder] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:37:05.603615 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9aa436a-6517-478c-9e96-eaa7bf6d879f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:37:05.621003 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9aa436a-6517-478c-9e96-eaa7bf6d879f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:37:05.623165 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9aa436a-6517-478c-9e96-eaa7bf6d879f service cinder] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:37:05.635793 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1169/2330] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Jun 15 10:37:05 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:37:06.138921 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dfd0b90-ee79-43d8-a173-f87eca7622af None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:37:06.213071 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dfd0b90-ee79-43d8-a173-f87eca7622af None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:37:06.218042 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75cc6da1-0de8-4ed7-b46b-8258f5e25c84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:37:06.218973 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cc6da1-0de8-4ed7-b46b-8258f5e25c84 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:37:06.219786 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cc6da1-0de8-4ed7-b46b-8258f5e25c84 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:37:06.220060 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cc6da1-0de8-4ed7-b46b-8258f5e25c84 admin admin] PATH_INFO: `/v3/users/73f5d8a1daab4cf0afde2a9b1c5616e5` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:37:06.221248 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75cc6da1-0de8-4ed7-b46b-8258f5e25c84 admin admin] RBAC: Authorizing `identity:delete_user(user_id=73f5d8a1daab4cf0afde2a9b1c5616e5)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:37:06.223251 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75cc6da1-0de8-4ed7-b46b-8258f5e25c84 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:37:06.250814 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:37:06.283030 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-75cc6da1-0de8-4ed7-b46b-8258f5e25c84 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '73f5d8a1daab4cf0afde2a9b1c5616e5'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:37:06.291761 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-75cc6da1-0de8-4ed7-b46b-8258f5e25c84 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '73f5d8a1daab4cf0afde2a9b1c5616e5'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:37:06.292604 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-75cc6da1-0de8-4ed7-b46b-8258f5e25c84 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '73f5d8a1daab4cf0afde2a9b1c5616e5'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:37:06.293237 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-75cc6da1-0de8-4ed7-b46b-8258f5e25c84 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '73f5d8a1daab4cf0afde2a9b1c5616e5'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:37:06.301874 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-75cc6da1-0de8-4ed7-b46b-8258f5e25c84 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '73f5d8a1daab4cf0afde2a9b1c5616e5'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:37:06.321244 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1162/2331] 10.222.0.49 () {58 vars in 1223 bytes} [Wed Jun 15 10:37:06 2022] DELETE /identity/v3/users/73f5d8a1daab4cf0afde2a9b1c5616e5 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:37:06.325954 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32fa7f1-6912-4c6d-82e6-8d988d330566 None None] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:37:06.326466 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32fa7f1-6912-4c6d-82e6-8d988d330566 None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:37:06.326565 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32fa7f1-6912-4c6d-82e6-8d988d330566 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:37:06.345714 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.auth.core [None req-d32fa7f1-6912-4c6d-82e6-8d988d330566 None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:37:06.346905 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-d32fa7f1-6912-4c6d-82e6-8d988d330566 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:37:06.358629 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:37:06.424348 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1170/2332] 10.222.0.49 () {58 vars in 957 bytes} [Wed Jun 15 10:37:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:37:06.434341 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75cc6da1-0de8-4ed7-b46b-8258f5e25c84 admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:37:06.435908 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-75cc6da1-0de8-4ed7-b46b-8258f5e25c84 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:37:06.507692 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75cc6da1-0de8-4ed7-b46b-8258f5e25c84 admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:37:06.511414 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4662aa0-57f3-4118-8a8b-ef910a4c8404 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0cb44a17d205431793e9a8791e477a32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '559b3e713f5844b6ae37151d56dd419f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:37:06.512069 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4662aa0-57f3-4118-8a8b-ef910a4c8404 service neutron] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:37:06.512352 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4662aa0-57f3-4118-8a8b-ef910a4c8404 service neutron] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:37:06.512628 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4662aa0-57f3-4118-8a8b-ef910a4c8404 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:37:06.524762 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4662aa0-57f3-4118-8a8b-ef910a4c8404 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:37:06.526485 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4662aa0-57f3-4118-8a8b-ef910a4c8404 service neutron] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:37:06.539684 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1163/2333] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Jun 15 10:37:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:37:06.933210 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32fa7f1-6912-4c6d-82e6-8d988d330566 None None] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:37:06.934799 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-d32fa7f1-6912-4c6d-82e6-8d988d330566 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:37:06.948486 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32fa7f1-6912-4c6d-82e6-8d988d330566 None None] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:37:06.970191 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8733b11f-732b-4a5b-b07c-05e48033e2f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:37:06.972138 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8733b11f-732b-4a5b-b07c-05e48033e2f5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:37:06.972868 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8733b11f-732b-4a5b-b07c-05e48033e2f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:37:06.973207 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8733b11f-732b-4a5b-b07c-05e48033e2f5 admin admin] PATH_INFO: `/v3/projects/b276120ccf5945a4983d2e98d3234bc5` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:37:06.975210 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8733b11f-732b-4a5b-b07c-05e48033e2f5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b276120ccf5945a4983d2e98d3234bc5)` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:37:06.977075 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8733b11f-732b-4a5b-b07c-05e48033e2f5 admin admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:37:07.050910 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-8733b11f-732b-4a5b-b07c-05e48033e2f5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b276120ccf5945a4983d2e98d3234bc5', 'request_id': 'req-8733b11f-732b-4a5b-b07c-05e48033e2f5'} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:37:07.059135 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-8733b11f-732b-4a5b-b07c-05e48033e2f5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b276120ccf5945a4983d2e98d3234bc5', 'request_id': 'req-8733b11f-732b-4a5b-b07c-05e48033e2f5'} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:37:07.065830 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-8733b11f-732b-4a5b-b07c-05e48033e2f5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b276120ccf5945a4983d2e98d3234bc5', 'request_id': 'req-8733b11f-732b-4a5b-b07c-05e48033e2f5'} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:37:07.078989 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-8733b11f-732b-4a5b-b07c-05e48033e2f5 admin admin] The token cache is being invalidate because project b276120ccf5945a4983d2e98d3234bc5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80119) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 10:37:07.079306 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-8733b11f-732b-4a5b-b07c-05e48033e2f5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:37:07.079963 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-8733b11f-732b-4a5b-b07c-05e48033e2f5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:37:07.081782 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1171/2334] 10.222.0.49 () {58 vars in 1232 bytes} [Wed Jun 15 10:37:06 2022] DELETE /identity/v3/projects/b276120ccf5945a4983d2e98d3234bc5 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:37:07.085587 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4662aa0-57f3-4118-8a8b-ef910a4c8404 service neutron] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:37:07.096887 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4662aa0-57f3-4118-8a8b-ef910a4c8404 service neutron] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:37:07.161654 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c12fedd-f4fb-490e-99a2-c0d61342c276 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:37:07.162662 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c12fedd-f4fb-490e-99a2-c0d61342c276 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:37:07.163040 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c12fedd-f4fb-490e-99a2-c0d61342c276 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:37:07.163401 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c12fedd-f4fb-490e-99a2-c0d61342c276 admin admin] PATH_INFO: `/v3/users/95f27e5f4a204fe48bea0fa1f7b89903` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:37:07.164682 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c12fedd-f4fb-490e-99a2-c0d61342c276 admin admin] RBAC: Authorizing `identity:delete_user(user_id=95f27e5f4a204fe48bea0fa1f7b89903)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:37:07.167265 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c12fedd-f4fb-490e-99a2-c0d61342c276 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:37:07.221586 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:37:07.223307 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-1c12fedd-f4fb-490e-99a2-c0d61342c276 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '95f27e5f4a204fe48bea0fa1f7b89903'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:37:07.242152 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-1c12fedd-f4fb-490e-99a2-c0d61342c276 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '95f27e5f4a204fe48bea0fa1f7b89903'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:37:07.243067 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-1c12fedd-f4fb-490e-99a2-c0d61342c276 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '95f27e5f4a204fe48bea0fa1f7b89903'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:37:07.243772 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-1c12fedd-f4fb-490e-99a2-c0d61342c276 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '95f27e5f4a204fe48bea0fa1f7b89903'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:37:07.252858 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-1c12fedd-f4fb-490e-99a2-c0d61342c276 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '95f27e5f4a204fe48bea0fa1f7b89903'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:37:07.277404 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1164/2335] 10.222.0.49 () {58 vars in 1223 bytes} [Wed Jun 15 10:37:07 2022] DELETE /identity/v3/users/95f27e5f4a204fe48bea0fa1f7b89903 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:37:07.606099 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8733b11f-732b-4a5b-b07c-05e48033e2f5 admin admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:37:07.607902 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-8733b11f-732b-4a5b-b07c-05e48033e2f5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:37:07.635208 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8733b11f-732b-4a5b-b07c-05e48033e2f5 admin admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:37:07.714431 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8efeca3-7e59-4a0e-aeba-02f2709c7a6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:37:07.715482 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8efeca3-7e59-4a0e-aeba-02f2709c7a6c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:37:07.715772 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8efeca3-7e59-4a0e-aeba-02f2709c7a6c admin admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:37:07.716045 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8efeca3-7e59-4a0e-aeba-02f2709c7a6c admin admin] PATH_INFO: `/v3/projects/bd5cba6852ed4e0f92c00b29b92b8764` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:37:07.717440 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8efeca3-7e59-4a0e-aeba-02f2709c7a6c admin admin] RBAC: Authorizing `identity:delete_project(project_id=bd5cba6852ed4e0f92c00b29b92b8764)` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:37:07.719444 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8efeca3-7e59-4a0e-aeba-02f2709c7a6c admin admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:37:07.768178 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:37:07.786381 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-f8efeca3-7e59-4a0e-aeba-02f2709c7a6c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bd5cba6852ed4e0f92c00b29b92b8764', 'request_id': 'req-f8efeca3-7e59-4a0e-aeba-02f2709c7a6c'} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:37:07.796938 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-f8efeca3-7e59-4a0e-aeba-02f2709c7a6c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bd5cba6852ed4e0f92c00b29b92b8764', 'request_id': 'req-f8efeca3-7e59-4a0e-aeba-02f2709c7a6c'} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:37:07.803981 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-f8efeca3-7e59-4a0e-aeba-02f2709c7a6c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bd5cba6852ed4e0f92c00b29b92b8764', 'request_id': 'req-f8efeca3-7e59-4a0e-aeba-02f2709c7a6c'} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:37:07.817810 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-f8efeca3-7e59-4a0e-aeba-02f2709c7a6c admin admin] The token cache is being invalidate because project bd5cba6852ed4e0f92c00b29b92b8764 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80119) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 10:37:07.818300 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-f8efeca3-7e59-4a0e-aeba-02f2709c7a6c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:37:07.819151 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-f8efeca3-7e59-4a0e-aeba-02f2709c7a6c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:37:07.820780 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1172/2336] 10.222.0.49 () {58 vars in 1232 bytes} [Wed Jun 15 10:37:07 2022] DELETE /identity/v3/projects/bd5cba6852ed4e0f92c00b29b92b8764 => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:37:07.825685 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22f8458-2d57-49e1-aed9-9a6864493879 None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:37:07.825973 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22f8458-2d57-49e1-aed9-9a6864493879 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:37:07.826252 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22f8458-2d57-49e1-aed9-9a6864493879 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:37:07.850790 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-b22f8458-2d57-49e1-aed9-9a6864493879 None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:37:07.852326 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-b22f8458-2d57-49e1-aed9-9a6864493879 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:37:07.929772 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1165/2337] 10.222.0.49 () {58 vars in 957 bytes} [Wed Jun 15 10:37:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:37:07.937729 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e606e30-da4e-475b-9513-9fab6f469090 None None] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:37:07.938023 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e606e30-da4e-475b-9513-9fab6f469090 None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:37:07.938307 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e606e30-da4e-475b-9513-9fab6f469090 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:37:07.958427 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.auth.core [None req-3e606e30-da4e-475b-9513-9fab6f469090 None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:37:07.959724 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-3e606e30-da4e-475b-9513-9fab6f469090 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:37:07.968193 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1173/2338] 10.222.0.49 () {58 vars in 957 bytes} [Wed Jun 15 10:37:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:37:07.972732 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22f8458-2d57-49e1-aed9-9a6864493879 None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:37:07.983073 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22f8458-2d57-49e1-aed9-9a6864493879 None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:37:07.988098 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b69ba6-c711-4ef5-84d9-99215b352ce5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:37:07.988802 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b69ba6-c711-4ef5-84d9-99215b352ce5 admin admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:37:07.989087 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b69ba6-c711-4ef5-84d9-99215b352ce5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:37:07.989365 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b69ba6-c711-4ef5-84d9-99215b352ce5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:37:07.990120 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b69ba6-c711-4ef5-84d9-99215b352ce5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:37:07.990433 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b69ba6-c711-4ef5-84d9-99215b352ce5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:37:07.992153 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b69ba6-c711-4ef5-84d9-99215b352ce5 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:37:08.006577 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1166/2339] 10.222.0.49 () {58 vars in 1152 bytes} [Wed Jun 15 10:37:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:37:08.011061 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e606e30-da4e-475b-9513-9fab6f469090 None None] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:37:08.023654 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e606e30-da4e-475b-9513-9fab6f469090 None None] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:37:08.032754 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63065ef8-fc79-4bfc-9c57-971d36e7c32f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:37:08.033493 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63065ef8-fc79-4bfc-9c57-971d36e7c32f admin admin] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:37:08.033773 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63065ef8-fc79-4bfc-9c57-971d36e7c32f admin admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:37:08.034053 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63065ef8-fc79-4bfc-9c57-971d36e7c32f admin admin] PATH_INFO: `/v3/projects` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:37:08.034777 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63065ef8-fc79-4bfc-9c57-971d36e7c32f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:37:08.041717 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63065ef8-fc79-4bfc-9c57-971d36e7c32f admin admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:37:08.085876 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1174/2340] 10.222.0.49 () {60 vars in 1152 bytes} [Wed Jun 15 10:37:08 2022] POST /identity/v3/projects => generated 377 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 10:37:08.087201 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b69ba6-c711-4ef5-84d9-99215b352ce5 admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:37:08.102908 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b69ba6-c711-4ef5-84d9-99215b352ce5 admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:37:08.174261 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744924ee-25b9-42b4-8884-fc11fba6b065 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:37:08.175546 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744924ee-25b9-42b4-8884-fc11fba6b065 admin admin] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:37:08.175546 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744924ee-25b9-42b4-8884-fc11fba6b065 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:37:08.175798 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744924ee-25b9-42b4-8884-fc11fba6b065 admin admin] PATH_INFO: `/v3/users` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:37:08.176535 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-744924ee-25b9-42b4-8884-fc11fba6b065 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:37:08.178553 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-744924ee-25b9-42b4-8884-fc11fba6b065 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:37:08.211197 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1167/2341] 10.222.0.49 () {60 vars in 1143 bytes} [Wed Jun 15 10:37:08 2022] POST /identity/v3/users => generated 289 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 10:37:08.214868 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63065ef8-fc79-4bfc-9c57-971d36e7c32f admin admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:37:08.235200 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63065ef8-fc79-4bfc-9c57-971d36e7c32f admin admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:37:08.242344 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7f0316-40a2-4408-9a47-9b483cb2fa3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:37:08.243350 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7f0316-40a2-4408-9a47-9b483cb2fa3e admin admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:37:08.243802 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7f0316-40a2-4408-9a47-9b483cb2fa3e admin admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:37:08.244228 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7f0316-40a2-4408-9a47-9b483cb2fa3e admin admin] PATH_INFO: `/v3/roles` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:37:08.245100 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e7f0316-40a2-4408-9a47-9b483cb2fa3e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:37:08.246858 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e7f0316-40a2-4408-9a47-9b483cb2fa3e admin admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:37:08.259944 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1175/2342] 10.222.0.49 () {58 vars in 1121 bytes} [Wed Jun 15 10:37:08 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 10:37:08.263920 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744924ee-25b9-42b4-8884-fc11fba6b065 admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:37:08.274791 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744924ee-25b9-42b4-8884-fc11fba6b065 admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:37:08.279874 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef80884f-a03e-43cb-9028-ac1d9d41a865 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:37:08.280591 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef80884f-a03e-43cb-9028-ac1d9d41a865 admin admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:37:08.281541 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef80884f-a03e-43cb-9028-ac1d9d41a865 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:37:08.281821 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef80884f-a03e-43cb-9028-ac1d9d41a865 admin admin] PATH_INFO: `/v3/roles` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:37:08.282558 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef80884f-a03e-43cb-9028-ac1d9d41a865 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:37:08.284254 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef80884f-a03e-43cb-9028-ac1d9d41a865 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:37:08.295701 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1168/2343] 10.222.0.49 () {58 vars in 1121 bytes} [Wed Jun 15 10:37:08 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 10:37:08.300060 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7f0316-40a2-4408-9a47-9b483cb2fa3e admin admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:37:08.314177 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7f0316-40a2-4408-9a47-9b483cb2fa3e admin admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:37:08.319754 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fea3a8e-5adf-454a-a5a1-10fa49e23bbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:37:08.320744 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fea3a8e-5adf-454a-a5a1-10fa49e23bbe admin admin] REQUEST_METHOD: `PUT` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:37:08.321104 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fea3a8e-5adf-454a-a5a1-10fa49e23bbe admin admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:37:08.321457 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fea3a8e-5adf-454a-a5a1-10fa49e23bbe admin admin] PATH_INFO: `/v3/projects/2f4cba5ef04b448ea912b222a6b67332/users/a2508cfe82f1445799b35f81c5ee5764/roles/6876e57cc51c448097ab109d35b2a3da` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:37:08.337557 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fea3a8e-5adf-454a-a5a1-10fa49e23bbe admin admin] RBAC: Authorizing `identity:create_grant(project_id=2f4cba5ef04b448ea912b222a6b67332, user_id=a2508cfe82f1445799b35f81c5ee5764, role_id=6876e57cc51c448097ab109d35b2a3da)` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:37:08.340333 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fea3a8e-5adf-454a-a5a1-10fa49e23bbe admin admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:37:08.355330 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1176/2344] 10.222.0.49 () {60 vars in 1482 bytes} [Wed Jun 15 10:37:08 2022] PUT /identity/v3/projects/2f4cba5ef04b448ea912b222a6b67332/users/a2508cfe82f1445799b35f81c5ee5764/roles/6876e57cc51c448097ab109d35b2a3da => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:37:08.367121 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ccad1f-4fbf-421a-96fb-656ce1912197 None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:37:08.367422 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ccad1f-4fbf-421a-96fb-656ce1912197 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:37:08.367701 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ccad1f-4fbf-421a-96fb-656ce1912197 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:37:08.401972 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-63ccad1f-4fbf-421a-96fb-656ce1912197 None None] MFA Rules not processed for user `a2508cfe82f1445799b35f81c5ee5764`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:37:08.403240 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-63ccad1f-4fbf-421a-96fb-656ce1912197 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:37:08.420584 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=93,keystone:INSERT=4,keystone:DELETE=3 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:37:08.475233 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1169/2345] 10.222.0.49 () {58 vars in 957 bytes} [Wed Jun 15 10:37:08 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 10:37:08.479857 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fea3a8e-5adf-454a-a5a1-10fa49e23bbe admin admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:37:08.553784 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fea3a8e-5adf-454a-a5a1-10fa49e23bbe admin admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:37:08.558667 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1febd3-bbf3-487c-8a54-2160aebb41c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:37:08.559415 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1febd3-bbf3-487c-8a54-2160aebb41c4 admin admin] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:37:08.564351 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1febd3-bbf3-487c-8a54-2160aebb41c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:37:08.564351 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1febd3-bbf3-487c-8a54-2160aebb41c4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:37:08.564351 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1febd3-bbf3-487c-8a54-2160aebb41c4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:37:08.564351 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1febd3-bbf3-487c-8a54-2160aebb41c4 admin admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:37:08.593711 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1177/2346] 10.222.0.49 () {60 vars in 1152 bytes} [Wed Jun 15 10:37:08 2022] POST /identity/v3/projects => generated 375 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 10:37:08.597449 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ccad1f-4fbf-421a-96fb-656ce1912197 None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:37:08.609472 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ccad1f-4fbf-421a-96fb-656ce1912197 None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:37:08.676968 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1d9e09-5a78-43ab-a9ee-aa3a315ed61a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:37:08.677672 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1d9e09-5a78-43ab-a9ee-aa3a315ed61a admin admin] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:37:08.677957 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1d9e09-5a78-43ab-a9ee-aa3a315ed61a admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:37:08.678235 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1d9e09-5a78-43ab-a9ee-aa3a315ed61a admin admin] PATH_INFO: `/v3/users` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:37:08.678964 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e1d9e09-5a78-43ab-a9ee-aa3a315ed61a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:37:08.680914 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e1d9e09-5a78-43ab-a9ee-aa3a315ed61a admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:37:08.703768 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1170/2347] 10.222.0.49 () {60 vars in 1143 bytes} [Wed Jun 15 10:37:08 2022] POST /identity/v3/users => generated 294 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 10:37:08.707531 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1febd3-bbf3-487c-8a54-2160aebb41c4 admin admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:37:08.719473 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1febd3-bbf3-487c-8a54-2160aebb41c4 admin admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:37:08.724285 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb047791-102e-4021-b35e-73ca8306f76f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:37:08.724950 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb047791-102e-4021-b35e-73ca8306f76f admin admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:37:08.725234 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb047791-102e-4021-b35e-73ca8306f76f admin admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:37:08.725512 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb047791-102e-4021-b35e-73ca8306f76f admin admin] PATH_INFO: `/v3/roles` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:37:08.726238 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb047791-102e-4021-b35e-73ca8306f76f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:37:08.727888 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb047791-102e-4021-b35e-73ca8306f76f admin admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:37:08.739268 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1178/2348] 10.222.0.49 () {58 vars in 1121 bytes} [Wed Jun 15 10:37:08 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 15 10:37:08.744015 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1d9e09-5a78-43ab-a9ee-aa3a315ed61a admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:37:08.755779 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1d9e09-5a78-43ab-a9ee-aa3a315ed61a admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:37:08.761483 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6346c712-295b-43c6-a046-29d6a80e7a0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:37:08.762575 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6346c712-295b-43c6-a046-29d6a80e7a0f admin admin] REQUEST_METHOD: `PUT` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:37:08.762860 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6346c712-295b-43c6-a046-29d6a80e7a0f admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:37:08.763173 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6346c712-295b-43c6-a046-29d6a80e7a0f admin admin] PATH_INFO: `/v3/projects/a9b7d8c1160049e9b06ef7c5d1ea898f/users/6066bbc3a04e41b0a321b49669565b89/roles/8051174e7a7947d5b991ebd793c2e543` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:37:08.797483 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6346c712-295b-43c6-a046-29d6a80e7a0f admin admin] RBAC: Authorizing `identity:create_grant(project_id=a9b7d8c1160049e9b06ef7c5d1ea898f, user_id=6066bbc3a04e41b0a321b49669565b89, role_id=8051174e7a7947d5b991ebd793c2e543)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:37:08.802903 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6346c712-295b-43c6-a046-29d6a80e7a0f admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:37:08.828471 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1171/2349] 10.222.0.49 () {60 vars in 1482 bytes} [Wed Jun 15 10:37:08 2022] PUT /identity/v3/projects/a9b7d8c1160049e9b06ef7c5d1ea898f/users/6066bbc3a04e41b0a321b49669565b89/roles/8051174e7a7947d5b991ebd793c2e543 => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:37:08.832258 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb047791-102e-4021-b35e-73ca8306f76f admin admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:37:08.842815 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb047791-102e-4021-b35e-73ca8306f76f admin admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:37:08.881681 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:37:08.903948 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f783ce0-a29f-4dfc-b85f-a4c7d0439dd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:37:08.904615 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f783ce0-a29f-4dfc-b85f-a4c7d0439dd0 admin admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:37:08.904896 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f783ce0-a29f-4dfc-b85f-a4c7d0439dd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:37:08.905171 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f783ce0-a29f-4dfc-b85f-a4c7d0439dd0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:37:08.905936 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f783ce0-a29f-4dfc-b85f-a4c7d0439dd0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:37:08.907577 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f783ce0-a29f-4dfc-b85f-a4c7d0439dd0 admin admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:37:08.920594 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1179/2350] 10.222.0.49 () {58 vars in 1121 bytes} [Wed Jun 15 10:37:08 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 10:37:08.922265 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6346c712-295b-43c6-a046-29d6a80e7a0f admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:37:08.932559 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6346c712-295b-43c6-a046-29d6a80e7a0f admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:37:08.937324 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e15a557-ff74-4cd7-9a10-70db30e1925a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:37:08.938231 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e15a557-ff74-4cd7-9a10-70db30e1925a admin admin] REQUEST_METHOD: `PUT` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:37:08.938507 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e15a557-ff74-4cd7-9a10-70db30e1925a admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:37:08.938782 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e15a557-ff74-4cd7-9a10-70db30e1925a admin admin] PATH_INFO: `/v3/projects/a9b7d8c1160049e9b06ef7c5d1ea898f/users/6066bbc3a04e41b0a321b49669565b89/roles/8051174e7a7947d5b991ebd793c2e543` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:37:08.940802 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e15a557-ff74-4cd7-9a10-70db30e1925a admin admin] RBAC: Authorizing `identity:create_grant(project_id=a9b7d8c1160049e9b06ef7c5d1ea898f, user_id=6066bbc3a04e41b0a321b49669565b89, role_id=8051174e7a7947d5b991ebd793c2e543)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:37:08.943461 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e15a557-ff74-4cd7-9a10-70db30e1925a admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:37:08.954832 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1172/2351] 10.222.0.49 () {60 vars in 1482 bytes} [Wed Jun 15 10:37:08 2022] PUT /identity/v3/projects/a9b7d8c1160049e9b06ef7c5d1ea898f/users/6066bbc3a04e41b0a321b49669565b89/roles/8051174e7a7947d5b991ebd793c2e543 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:37:08.964908 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e36f78-78b5-4c51-b866-7b13235aabed None None] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:37:08.965202 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e36f78-78b5-4c51-b866-7b13235aabed None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:37:08.965483 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e36f78-78b5-4c51-b866-7b13235aabed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:37:08.991618 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.auth.core [None req-a4e36f78-78b5-4c51-b866-7b13235aabed None None] MFA Rules not processed for user `6066bbc3a04e41b0a321b49669565b89`. Rule list: `[]` (Enabled: `True`). {{(pid=80119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:37:08.992821 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-a4e36f78-78b5-4c51-b866-7b13235aabed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:37:09.059717 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1180/2352] 10.222.0.49 () {58 vars in 957 bytes} [Wed Jun 15 10:37:08 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:37:09.070710 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e15a557-ff74-4cd7-9a10-70db30e1925a admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:37:09.072496 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-3e15a557-ff74-4cd7-9a10-70db30e1925a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:37:09.130039 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e15a557-ff74-4cd7-9a10-70db30e1925a admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:37:09.133656 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebfec60a-35dd-4cf6-925f-2676f355701f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20d1327553404b80aecee87c78fcb52b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '559b3e713f5844b6ae37151d56dd419f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:37:09.134300 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfec60a-35dd-4cf6-925f-2676f355701f service cinder] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:37:09.134590 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfec60a-35dd-4cf6-925f-2676f355701f service cinder] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:37:09.134870 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfec60a-35dd-4cf6-925f-2676f355701f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:37:09.160157 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:37:09.192074 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebfec60a-35dd-4cf6-925f-2676f355701f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:37:09.193508 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebfec60a-35dd-4cf6-925f-2676f355701f service cinder] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:37:09.203039 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1173/2353] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Jun 15 10:37:09 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:37:13.807863 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e36f78-78b5-4c51-b866-7b13235aabed None None] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:37:13.818595 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e36f78-78b5-4c51-b866-7b13235aabed None None] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:37:13.822318 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af7d365-ca51-49a6-be78-702997084cd8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20d1327553404b80aecee87c78fcb52b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '559b3e713f5844b6ae37151d56dd419f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:37:13.823558 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af7d365-ca51-49a6-be78-702997084cd8 service cinder] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:37:13.823558 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af7d365-ca51-49a6-be78-702997084cd8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:37:13.823818 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af7d365-ca51-49a6-be78-702997084cd8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:37:13.835976 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af7d365-ca51-49a6-be78-702997084cd8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:37:13.837999 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af7d365-ca51-49a6-be78-702997084cd8 service cinder] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:37:13.850535 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1181/2354] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Jun 15 10:37:13 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:37:16.798459 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebfec60a-35dd-4cf6-925f-2676f355701f service cinder] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:37:16.799875 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-ebfec60a-35dd-4cf6-925f-2676f355701f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:37:16.871793 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebfec60a-35dd-4cf6-925f-2676f355701f service cinder] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:37:16.877732 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27bc7ef-cb07-4129-aab2-9715dbb87fed service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ac54528370d4747b6e8777fba9d0c82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '559b3e713f5844b6ae37151d56dd419f', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:37:16.878370 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27bc7ef-cb07-4129-aab2-9715dbb87fed service nova] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:37:16.878653 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27bc7ef-cb07-4129-aab2-9715dbb87fed service nova] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:37:16.878927 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27bc7ef-cb07-4129-aab2-9715dbb87fed service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:37:16.880709 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-f27bc7ef-cb07-4129-aab2-9715dbb87fed service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:37:16.937323 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f27bc7ef-cb07-4129-aab2-9715dbb87fed service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:37:16.938890 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f27bc7ef-cb07-4129-aab2-9715dbb87fed service nova] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:37:16.947690 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1174/2355] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Jun 15 10:37:16 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:37:17.469662 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6c38e9-fafe-4752-bfba-16104d72a3e1 None None] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:37:17.470184 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6c38e9-fafe-4752-bfba-16104d72a3e1 None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:37:17.470384 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6c38e9-fafe-4752-bfba-16104d72a3e1 None None] PATH_INFO: `/` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:37:17.471930 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1182/2356] 10.222.0.49 () {54 vars in 921 bytes} [Wed Jun 15 10:37:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:37:17.478383 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4712263f-3746-4eda-840d-db7e60c0530d None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:37:17.478752 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4712263f-3746-4eda-840d-db7e60c0530d None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:37:17.479127 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4712263f-3746-4eda-840d-db7e60c0530d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:37:17.516634 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-4712263f-3746-4eda-840d-db7e60c0530d None None] MFA Rules not processed for user `5ac54528370d4747b6e8777fba9d0c82`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:37:17.517852 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-4712263f-3746-4eda-840d-db7e60c0530d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:37:17.528196 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1175/2357] 10.222.0.49 () {58 vars in 1018 bytes} [Wed Jun 15 10:37:17 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:37:17.540045 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6c38e9-fafe-4752-bfba-16104d72a3e1 None None] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:37:17.541836 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-ec6c38e9-fafe-4752-bfba-16104d72a3e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:37:17.620033 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6c38e9-fafe-4752-bfba-16104d72a3e1 None None] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:37:17.623639 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4592ab-6002-4dec-bc47-fad8ddd8033a service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91f2fe3be62e49548eae208e57bf2a93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '559b3e713f5844b6ae37151d56dd419f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:37:17.624305 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4592ab-6002-4dec-bc47-fad8ddd8033a service glance] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:37:17.624587 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4592ab-6002-4dec-bc47-fad8ddd8033a service glance] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:37:17.624865 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4592ab-6002-4dec-bc47-fad8ddd8033a service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:37:17.634751 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf4592ab-6002-4dec-bc47-fad8ddd8033a service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:37:17.636216 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf4592ab-6002-4dec-bc47-fad8ddd8033a service glance] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:37:17.646591 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1183/2358] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Jun 15 10:37:17 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:37:17.712382 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987407e3-0957-4395-83a1-2ed3f46c1ca4 None None] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:37:17.712675 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987407e3-0957-4395-83a1-2ed3f46c1ca4 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:37:17.712959 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987407e3-0957-4395-83a1-2ed3f46c1ca4 None None] PATH_INFO: `/` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:37:17.714339 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1176/2359] 10.222.0.49 () {54 vars in 921 bytes} [Wed Jun 15 10:37:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:37:17.719774 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c71e0b-4ea7-475b-a839-3a7e04738060 None None] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:37:17.720089 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c71e0b-4ea7-475b-a839-3a7e04738060 None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:37:17.720354 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c71e0b-4ea7-475b-a839-3a7e04738060 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:37:17.739701 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.auth.core [None req-70c71e0b-4ea7-475b-a839-3a7e04738060 None None] MFA Rules not processed for user `0cb44a17d205431793e9a8791e477a32`. Rule list: `[]` (Enabled: `True`). {{(pid=80119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:37:17.740940 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-70c71e0b-4ea7-475b-a839-3a7e04738060 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:37:17.773326 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=100 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:37:17.806166 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1184/2360] 10.222.0.49 () {58 vars in 1018 bytes} [Wed Jun 15 10:37:17 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:37:17.818502 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987407e3-0957-4395-83a1-2ed3f46c1ca4 None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:37:17.820341 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-987407e3-0957-4395-83a1-2ed3f46c1ca4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:37:17.835258 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987407e3-0957-4395-83a1-2ed3f46c1ca4 None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:37:17.839139 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b870b4bf-ce25-4cf4-8377-39d768057368 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0cb44a17d205431793e9a8791e477a32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '559b3e713f5844b6ae37151d56dd419f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:37:17.839862 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b870b4bf-ce25-4cf4-8377-39d768057368 service neutron] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:37:17.840225 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b870b4bf-ce25-4cf4-8377-39d768057368 service neutron] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:37:17.840515 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b870b4bf-ce25-4cf4-8377-39d768057368 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:37:17.848649 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b870b4bf-ce25-4cf4-8377-39d768057368 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:37:17.850096 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b870b4bf-ce25-4cf4-8377-39d768057368 service neutron] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:37:17.857842 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1177/2361] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Jun 15 10:37:17 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:37:18.147198 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791693d8-6051-4007-a8a1-d4683eada52b None None] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:37:18.147486 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791693d8-6051-4007-a8a1-d4683eada52b None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:37:18.147762 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791693d8-6051-4007-a8a1-d4683eada52b None None] PATH_INFO: `/` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:37:18.149143 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1185/2362] 10.222.0.49 () {54 vars in 921 bytes} [Wed Jun 15 10:37:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:37:18.154282 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad37603-328b-4d27-baca-de5e87062344 None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:37:18.154573 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad37603-328b-4d27-baca-de5e87062344 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:37:18.154856 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad37603-328b-4d27-baca-de5e87062344 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:37:18.174393 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-cad37603-328b-4d27-baca-de5e87062344 None None] MFA Rules not processed for user `5ac54528370d4747b6e8777fba9d0c82`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:37:18.175640 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-cad37603-328b-4d27-baca-de5e87062344 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:37:18.184822 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1178/2363] 10.222.0.49 () {58 vars in 1018 bytes} [Wed Jun 15 10:37:18 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:37:18.194748 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791693d8-6051-4007-a8a1-d4683eada52b None None] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:37:18.205850 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791693d8-6051-4007-a8a1-d4683eada52b None None] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:37:18.209269 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11f4bb6-05e3-44ee-93b2-5d64d3e4fa0c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0cb44a17d205431793e9a8791e477a32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '559b3e713f5844b6ae37151d56dd419f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:37:18.209895 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11f4bb6-05e3-44ee-93b2-5d64d3e4fa0c service neutron] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:37:18.210176 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11f4bb6-05e3-44ee-93b2-5d64d3e4fa0c service neutron] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:37:18.210450 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11f4bb6-05e3-44ee-93b2-5d64d3e4fa0c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:37:18.220974 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e11f4bb6-05e3-44ee-93b2-5d64d3e4fa0c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:37:18.222398 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e11f4bb6-05e3-44ee-93b2-5d64d3e4fa0c service neutron] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:37:18.232798 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1186/2364] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Jun 15 10:37:18 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:37:22.710115 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29121c00-66bb-4480-b34b-1df2deaabd33 None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:37:22.710581 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29121c00-66bb-4480-b34b-1df2deaabd33 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:37:22.710707 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29121c00-66bb-4480-b34b-1df2deaabd33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:37:22.727948 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-29121c00-66bb-4480-b34b-1df2deaabd33 None None] MFA Rules not processed for user `103e10244eca444b86b9c51ff314d445`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:37:22.729149 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-29121c00-66bb-4480-b34b-1df2deaabd33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:37:22.745970 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:37:22.784886 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1179/2365] 10.222.0.49 () {58 vars in 1009 bytes} [Wed Jun 15 10:37:22 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:37:22.806347 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11f4bb6-05e3-44ee-93b2-5d64d3e4fa0c service neutron] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:37:22.807761 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-e11f4bb6-05e3-44ee-93b2-5d64d3e4fa0c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:37:22.877894 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11f4bb6-05e3-44ee-93b2-5d64d3e4fa0c service neutron] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:37:22.883337 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab14ee74-4fa8-47e1-a324-5288ba990c6e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a69f99210444f9696e8b22615fb4a41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '559b3e713f5844b6ae37151d56dd419f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:37:22.884019 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab14ee74-4fa8-47e1-a324-5288ba990c6e service swift] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:37:22.884323 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab14ee74-4fa8-47e1-a324-5288ba990c6e service swift] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:37:22.884602 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab14ee74-4fa8-47e1-a324-5288ba990c6e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:37:22.893234 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab14ee74-4fa8-47e1-a324-5288ba990c6e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:37:22.894971 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab14ee74-4fa8-47e1-a324-5288ba990c6e service swift] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:37:22.903407 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1187/2366] 10.222.0.49 () {60 vars in 1384 bytes} [Wed Jun 15 10:37:22 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 15 10:37:32.777504 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=15 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:37:32.899851 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=47 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:37:38.857757 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29121c00-66bb-4480-b34b-1df2deaabd33 None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:37:38.919885 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29121c00-66bb-4480-b34b-1df2deaabd33 None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:37:38.924857 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5f8171-2cee-4294-8e1b-f25c9dbc3f38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:37:38.925720 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5f8171-2cee-4294-8e1b-f25c9dbc3f38 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:37:38.926004 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5f8171-2cee-4294-8e1b-f25c9dbc3f38 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:37:38.926311 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5f8171-2cee-4294-8e1b-f25c9dbc3f38 admin admin] PATH_INFO: `/v3/users/a2508cfe82f1445799b35f81c5ee5764` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:37:38.927530 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5f8171-2cee-4294-8e1b-f25c9dbc3f38 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a2508cfe82f1445799b35f81c5ee5764)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:37:38.929450 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5f8171-2cee-4294-8e1b-f25c9dbc3f38 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:37:38.974881 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-0d5f8171-2cee-4294-8e1b-f25c9dbc3f38 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a2508cfe82f1445799b35f81c5ee5764'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:37:38.981820 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-0d5f8171-2cee-4294-8e1b-f25c9dbc3f38 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a2508cfe82f1445799b35f81c5ee5764'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:37:38.982477 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-0d5f8171-2cee-4294-8e1b-f25c9dbc3f38 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a2508cfe82f1445799b35f81c5ee5764'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:37:38.983081 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-0d5f8171-2cee-4294-8e1b-f25c9dbc3f38 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a2508cfe82f1445799b35f81c5ee5764'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:37:38.990881 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-0d5f8171-2cee-4294-8e1b-f25c9dbc3f38 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a2508cfe82f1445799b35f81c5ee5764'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:37:39.008009 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1180/2367] 10.222.0.49 () {58 vars in 1223 bytes} [Wed Jun 15 10:37:38 2022] DELETE /identity/v3/users/a2508cfe82f1445799b35f81c5ee5764 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:37:39.012542 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49f2f78-a421-4801-8333-ef4699904324 None None] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:37:39.013028 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49f2f78-a421-4801-8333-ef4699904324 None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:37:39.013134 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49f2f78-a421-4801-8333-ef4699904324 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:37:39.029917 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.auth.core [None req-a49f2f78-a421-4801-8333-ef4699904324 None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:37:39.031160 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-a49f2f78-a421-4801-8333-ef4699904324 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:37:39.088962 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1188/2368] 10.222.0.49 () {58 vars in 957 bytes} [Wed Jun 15 10:37:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:37:39.098477 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5f8171-2cee-4294-8e1b-f25c9dbc3f38 admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:37:39.099963 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-0d5f8171-2cee-4294-8e1b-f25c9dbc3f38 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:37:39.158094 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5f8171-2cee-4294-8e1b-f25c9dbc3f38 admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:37:39.162017 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3791c1c7-0380-4950-8f2b-873082193857 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0cb44a17d205431793e9a8791e477a32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '559b3e713f5844b6ae37151d56dd419f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:37:39.162675 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3791c1c7-0380-4950-8f2b-873082193857 service neutron] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:37:39.162963 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3791c1c7-0380-4950-8f2b-873082193857 service neutron] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:37:39.163292 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3791c1c7-0380-4950-8f2b-873082193857 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:37:39.172989 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3791c1c7-0380-4950-8f2b-873082193857 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:37:39.174916 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3791c1c7-0380-4950-8f2b-873082193857 service neutron] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:37:39.184701 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1181/2369] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Jun 15 10:37:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:37:39.475959 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49f2f78-a421-4801-8333-ef4699904324 None None] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:37:39.477682 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-a49f2f78-a421-4801-8333-ef4699904324 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:37:39.494036 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49f2f78-a421-4801-8333-ef4699904324 None None] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:37:39.498818 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5120ea0c-e3b8-4274-8920-cfd29a6c9ce9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:37:39.499896 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5120ea0c-e3b8-4274-8920-cfd29a6c9ce9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:37:39.500233 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5120ea0c-e3b8-4274-8920-cfd29a6c9ce9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:37:39.500576 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5120ea0c-e3b8-4274-8920-cfd29a6c9ce9 admin admin] PATH_INFO: `/v3/projects/2f4cba5ef04b448ea912b222a6b67332` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:37:39.501836 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5120ea0c-e3b8-4274-8920-cfd29a6c9ce9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2f4cba5ef04b448ea912b222a6b67332)` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:37:39.503902 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5120ea0c-e3b8-4274-8920-cfd29a6c9ce9 admin admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:37:39.552081 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-5120ea0c-e3b8-4274-8920-cfd29a6c9ce9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2f4cba5ef04b448ea912b222a6b67332', 'request_id': 'req-5120ea0c-e3b8-4274-8920-cfd29a6c9ce9'} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:37:39.558683 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-5120ea0c-e3b8-4274-8920-cfd29a6c9ce9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2f4cba5ef04b448ea912b222a6b67332', 'request_id': 'req-5120ea0c-e3b8-4274-8920-cfd29a6c9ce9'} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:37:39.564156 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-5120ea0c-e3b8-4274-8920-cfd29a6c9ce9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2f4cba5ef04b448ea912b222a6b67332', 'request_id': 'req-5120ea0c-e3b8-4274-8920-cfd29a6c9ce9'} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:37:39.574923 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-5120ea0c-e3b8-4274-8920-cfd29a6c9ce9 admin admin] The token cache is being invalidate because project 2f4cba5ef04b448ea912b222a6b67332 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80119) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 10:37:39.575256 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-5120ea0c-e3b8-4274-8920-cfd29a6c9ce9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:37:39.575973 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-5120ea0c-e3b8-4274-8920-cfd29a6c9ce9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:37:39.577580 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1189/2370] 10.222.0.49 () {58 vars in 1232 bytes} [Wed Jun 15 10:37:39 2022] DELETE /identity/v3/projects/2f4cba5ef04b448ea912b222a6b67332 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 10:37:39.581965 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3791c1c7-0380-4950-8f2b-873082193857 service neutron] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:37:39.591621 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3791c1c7-0380-4950-8f2b-873082193857 service neutron] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:37:39.610508 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:37:39.650276 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f831d3-5749-45b3-a0d7-11116b7d183c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:37:39.651440 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f831d3-5749-45b3-a0d7-11116b7d183c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:37:39.651727 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f831d3-5749-45b3-a0d7-11116b7d183c admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:37:39.652005 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f831d3-5749-45b3-a0d7-11116b7d183c admin admin] PATH_INFO: `/v3/users/6066bbc3a04e41b0a321b49669565b89` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:37:39.653209 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f831d3-5749-45b3-a0d7-11116b7d183c admin admin] RBAC: Authorizing `identity:delete_user(user_id=6066bbc3a04e41b0a321b49669565b89)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:37:39.655111 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f831d3-5749-45b3-a0d7-11116b7d183c admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:37:39.699176 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-86f831d3-5749-45b3-a0d7-11116b7d183c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6066bbc3a04e41b0a321b49669565b89'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:37:39.706414 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-86f831d3-5749-45b3-a0d7-11116b7d183c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6066bbc3a04e41b0a321b49669565b89'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:37:39.707114 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-86f831d3-5749-45b3-a0d7-11116b7d183c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6066bbc3a04e41b0a321b49669565b89'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:37:39.707735 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-86f831d3-5749-45b3-a0d7-11116b7d183c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6066bbc3a04e41b0a321b49669565b89'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:37:39.715579 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-86f831d3-5749-45b3-a0d7-11116b7d183c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6066bbc3a04e41b0a321b49669565b89'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:37:39.733594 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1182/2371] 10.222.0.49 () {58 vars in 1223 bytes} [Wed Jun 15 10:37:39 2022] DELETE /identity/v3/users/6066bbc3a04e41b0a321b49669565b89 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:37:40.064337 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5120ea0c-e3b8-4274-8920-cfd29a6c9ce9 admin admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:37:40.066727 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-5120ea0c-e3b8-4274-8920-cfd29a6c9ce9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:37:40.079805 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5120ea0c-e3b8-4274-8920-cfd29a6c9ce9 admin admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:37:40.139868 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a606b9c-bfda-45ea-9247-5c1331ce2ac9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:37:40.140726 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a606b9c-bfda-45ea-9247-5c1331ce2ac9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:37:40.141006 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a606b9c-bfda-45ea-9247-5c1331ce2ac9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:37:40.141278 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a606b9c-bfda-45ea-9247-5c1331ce2ac9 admin admin] PATH_INFO: `/v3/projects/a9b7d8c1160049e9b06ef7c5d1ea898f` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:37:40.142441 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a606b9c-bfda-45ea-9247-5c1331ce2ac9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a9b7d8c1160049e9b06ef7c5d1ea898f)` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:37:40.144926 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a606b9c-bfda-45ea-9247-5c1331ce2ac9 admin admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:37:40.174208 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:37:40.191608 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-9a606b9c-bfda-45ea-9247-5c1331ce2ac9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a9b7d8c1160049e9b06ef7c5d1ea898f', 'request_id': 'req-9a606b9c-bfda-45ea-9247-5c1331ce2ac9'} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:37:40.198374 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-9a606b9c-bfda-45ea-9247-5c1331ce2ac9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a9b7d8c1160049e9b06ef7c5d1ea898f', 'request_id': 'req-9a606b9c-bfda-45ea-9247-5c1331ce2ac9'} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:37:40.203824 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-9a606b9c-bfda-45ea-9247-5c1331ce2ac9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a9b7d8c1160049e9b06ef7c5d1ea898f', 'request_id': 'req-9a606b9c-bfda-45ea-9247-5c1331ce2ac9'} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:37:40.215313 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-9a606b9c-bfda-45ea-9247-5c1331ce2ac9 admin admin] The token cache is being invalidate because project a9b7d8c1160049e9b06ef7c5d1ea898f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80119) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 10:37:40.215602 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-9a606b9c-bfda-45ea-9247-5c1331ce2ac9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:37:40.216234 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-9a606b9c-bfda-45ea-9247-5c1331ce2ac9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:37:40.217807 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1190/2372] 10.222.0.49 () {58 vars in 1232 bytes} [Wed Jun 15 10:37:40 2022] DELETE /identity/v3/projects/a9b7d8c1160049e9b06ef7c5d1ea898f => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:37:40.222754 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7c4ebd-812f-4161-a2b8-41d93f00003f None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:37:40.223280 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7c4ebd-812f-4161-a2b8-41d93f00003f None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:37:40.223397 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7c4ebd-812f-4161-a2b8-41d93f00003f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:37:40.240665 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-8b7c4ebd-812f-4161-a2b8-41d93f00003f None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:37:40.241792 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-8b7c4ebd-812f-4161-a2b8-41d93f00003f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:37:40.305108 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1183/2373] 10.222.0.49 () {58 vars in 957 bytes} [Wed Jun 15 10:37:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:37:40.313911 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d80059-b2b5-4eb8-a6b2-96d78b705128 None None] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:37:40.314205 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d80059-b2b5-4eb8-a6b2-96d78b705128 None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:37:40.314489 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d80059-b2b5-4eb8-a6b2-96d78b705128 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:37:40.333094 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.auth.core [None req-87d80059-b2b5-4eb8-a6b2-96d78b705128 None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:37:40.334291 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-87d80059-b2b5-4eb8-a6b2-96d78b705128 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:37:40.342084 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1191/2374] 10.222.0.49 () {58 vars in 957 bytes} [Wed Jun 15 10:37:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:37:40.345969 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7c4ebd-812f-4161-a2b8-41d93f00003f None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:37:40.355621 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7c4ebd-812f-4161-a2b8-41d93f00003f None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:37:40.360229 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a3beef-70d5-4229-a7cf-b163a28cf16f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:37:40.360884 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a3beef-70d5-4229-a7cf-b163a28cf16f admin admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:37:40.361200 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a3beef-70d5-4229-a7cf-b163a28cf16f admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:37:40.361478 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a3beef-70d5-4229-a7cf-b163a28cf16f admin admin] PATH_INFO: `/v3/domains` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:37:40.362728 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a3beef-70d5-4229-a7cf-b163a28cf16f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:37:40.363034 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a3beef-70d5-4229-a7cf-b163a28cf16f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:37:40.364656 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a3beef-70d5-4229-a7cf-b163a28cf16f admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:37:40.376058 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1184/2375] 10.222.0.49 () {58 vars in 1152 bytes} [Wed Jun 15 10:37:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:37:40.380873 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d80059-b2b5-4eb8-a6b2-96d78b705128 None None] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:37:40.389890 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d80059-b2b5-4eb8-a6b2-96d78b705128 None None] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:37:40.394624 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa894d2f-08c6-45c7-83aa-58d72c671e19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:37:40.395299 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa894d2f-08c6-45c7-83aa-58d72c671e19 admin admin] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:37:40.395586 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa894d2f-08c6-45c7-83aa-58d72c671e19 admin admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:37:40.395859 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa894d2f-08c6-45c7-83aa-58d72c671e19 admin admin] PATH_INFO: `/v3/projects` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:37:40.396540 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa894d2f-08c6-45c7-83aa-58d72c671e19 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:37:40.398433 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa894d2f-08c6-45c7-83aa-58d72c671e19 admin admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:37:40.433267 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1192/2376] 10.222.0.49 () {60 vars in 1152 bytes} [Wed Jun 15 10:37:40 2022] POST /identity/v3/projects => generated 397 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 10:37:40.437201 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a3beef-70d5-4229-a7cf-b163a28cf16f admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:37:40.447541 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a3beef-70d5-4229-a7cf-b163a28cf16f admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:37:40.473820 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:37:40.508147 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49dd3a6-e756-4b81-a3fc-ff5f0317434a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:37:40.508750 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49dd3a6-e756-4b81-a3fc-ff5f0317434a admin admin] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:37:40.509035 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49dd3a6-e756-4b81-a3fc-ff5f0317434a admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:37:40.509311 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49dd3a6-e756-4b81-a3fc-ff5f0317434a admin admin] PATH_INFO: `/v3/users` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:37:40.510000 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49dd3a6-e756-4b81-a3fc-ff5f0317434a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:37:40.511985 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49dd3a6-e756-4b81-a3fc-ff5f0317434a admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:37:40.531373 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1185/2377] 10.222.0.49 () {60 vars in 1143 bytes} [Wed Jun 15 10:37:40 2022] POST /identity/v3/users => generated 299 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 10:37:40.535358 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa894d2f-08c6-45c7-83aa-58d72c671e19 admin admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:37:40.545694 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa894d2f-08c6-45c7-83aa-58d72c671e19 admin admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:37:40.550858 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d831f4-4bb0-4f9d-9999-5cdff0fe048f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:37:40.551554 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d831f4-4bb0-4f9d-9999-5cdff0fe048f admin admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:37:40.551835 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d831f4-4bb0-4f9d-9999-5cdff0fe048f admin admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:37:40.552108 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d831f4-4bb0-4f9d-9999-5cdff0fe048f admin admin] PATH_INFO: `/v3/roles` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:37:40.552829 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d831f4-4bb0-4f9d-9999-5cdff0fe048f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:37:40.554499 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d831f4-4bb0-4f9d-9999-5cdff0fe048f admin admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:37:40.567764 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1193/2378] 10.222.0.49 () {58 vars in 1121 bytes} [Wed Jun 15 10:37:40 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 10:37:40.568962 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49dd3a6-e756-4b81-a3fc-ff5f0317434a admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:37:40.579352 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49dd3a6-e756-4b81-a3fc-ff5f0317434a admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:37:40.584001 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504d7dc8-e655-432c-95d6-d9f84b9e8b90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:37:40.584659 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504d7dc8-e655-432c-95d6-d9f84b9e8b90 admin admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:37:40.584957 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504d7dc8-e655-432c-95d6-d9f84b9e8b90 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:37:40.585214 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504d7dc8-e655-432c-95d6-d9f84b9e8b90 admin admin] PATH_INFO: `/v3/roles` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:37:40.585936 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-504d7dc8-e655-432c-95d6-d9f84b9e8b90 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:37:40.587569 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-504d7dc8-e655-432c-95d6-d9f84b9e8b90 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:37:40.598686 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1186/2379] 10.222.0.49 () {58 vars in 1121 bytes} [Wed Jun 15 10:37:40 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 10:37:40.602798 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d831f4-4bb0-4f9d-9999-5cdff0fe048f admin admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:37:40.613390 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d831f4-4bb0-4f9d-9999-5cdff0fe048f admin admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:37:40.618143 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0838004-422a-4c96-b079-e2fbb3fbca90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:37:40.619044 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0838004-422a-4c96-b079-e2fbb3fbca90 admin admin] REQUEST_METHOD: `PUT` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:37:40.619339 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0838004-422a-4c96-b079-e2fbb3fbca90 admin admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:37:40.619611 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0838004-422a-4c96-b079-e2fbb3fbca90 admin admin] PATH_INFO: `/v3/projects/e7159500a9864347a625ca47a937416f/users/c861f650bd0b4df897d8bd9389ad967c/roles/6876e57cc51c448097ab109d35b2a3da` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:37:40.634895 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0838004-422a-4c96-b079-e2fbb3fbca90 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e7159500a9864347a625ca47a937416f, user_id=c861f650bd0b4df897d8bd9389ad967c, role_id=6876e57cc51c448097ab109d35b2a3da)` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:37:40.637594 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0838004-422a-4c96-b079-e2fbb3fbca90 admin admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:37:40.650302 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1194/2380] 10.222.0.49 () {60 vars in 1482 bytes} [Wed Jun 15 10:37:40 2022] PUT /identity/v3/projects/e7159500a9864347a625ca47a937416f/users/c861f650bd0b4df897d8bd9389ad967c/roles/6876e57cc51c448097ab109d35b2a3da => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:37:40.660433 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1995dfdf-a9b1-4b3a-aca4-99bbc6b45283 None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:37:40.660727 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1995dfdf-a9b1-4b3a-aca4-99bbc6b45283 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:37:40.661004 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1995dfdf-a9b1-4b3a-aca4-99bbc6b45283 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:37:40.687801 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-1995dfdf-a9b1-4b3a-aca4-99bbc6b45283 None None] MFA Rules not processed for user `c861f650bd0b4df897d8bd9389ad967c`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:37:40.689033 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-1995dfdf-a9b1-4b3a-aca4-99bbc6b45283 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:37:40.752957 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1187/2381] 10.222.0.49 () {58 vars in 957 bytes} [Wed Jun 15 10:37:40 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:37:40.757275 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0838004-422a-4c96-b079-e2fbb3fbca90 admin admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:37:40.818204 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0838004-422a-4c96-b079-e2fbb3fbca90 admin admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:37:40.822980 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e618181-8abb-4ef5-b69f-7354edee6f88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:37:40.823663 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e618181-8abb-4ef5-b69f-7354edee6f88 admin admin] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:37:40.823945 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e618181-8abb-4ef5-b69f-7354edee6f88 admin admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:37:40.824220 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e618181-8abb-4ef5-b69f-7354edee6f88 admin admin] PATH_INFO: `/v3/projects` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:37:40.824941 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e618181-8abb-4ef5-b69f-7354edee6f88 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:37:40.826870 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e618181-8abb-4ef5-b69f-7354edee6f88 admin admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:37:40.851877 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1195/2382] 10.222.0.49 () {60 vars in 1152 bytes} [Wed Jun 15 10:37:40 2022] POST /identity/v3/projects => generated 401 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 10:37:40.855923 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1995dfdf-a9b1-4b3a-aca4-99bbc6b45283 None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:37:40.867300 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1995dfdf-a9b1-4b3a-aca4-99bbc6b45283 None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:37:40.923656 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b250063f-04d8-46d0-9f43-cce9efb2303b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:37:40.924374 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b250063f-04d8-46d0-9f43-cce9efb2303b admin admin] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:37:40.925150 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b250063f-04d8-46d0-9f43-cce9efb2303b admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:37:40.925423 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b250063f-04d8-46d0-9f43-cce9efb2303b admin admin] PATH_INFO: `/v3/users` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:37:40.926114 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b250063f-04d8-46d0-9f43-cce9efb2303b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:37:40.928056 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b250063f-04d8-46d0-9f43-cce9efb2303b admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:37:40.948819 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1188/2383] 10.222.0.49 () {60 vars in 1143 bytes} [Wed Jun 15 10:37:40 2022] POST /identity/v3/users => generated 307 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 10:37:40.951251 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e618181-8abb-4ef5-b69f-7354edee6f88 admin admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:37:40.962361 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e618181-8abb-4ef5-b69f-7354edee6f88 admin admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:37:40.967392 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8adcddf-56b9-4bd8-8831-9617df303dae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:37:40.968072 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8adcddf-56b9-4bd8-8831-9617df303dae admin admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:37:40.968360 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8adcddf-56b9-4bd8-8831-9617df303dae admin admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:37:40.968641 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8adcddf-56b9-4bd8-8831-9617df303dae admin admin] PATH_INFO: `/v3/roles` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:37:40.969368 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8adcddf-56b9-4bd8-8831-9617df303dae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:37:40.970984 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8adcddf-56b9-4bd8-8831-9617df303dae admin admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:37:40.982396 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1196/2384] 10.222.0.49 () {58 vars in 1121 bytes} [Wed Jun 15 10:37:40 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 10:37:40.986516 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b250063f-04d8-46d0-9f43-cce9efb2303b admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:37:40.997563 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b250063f-04d8-46d0-9f43-cce9efb2303b admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:37:41.002268 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1931da6b-bd6d-431b-8876-eb60a92b1222 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:37:41.003513 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1931da6b-bd6d-431b-8876-eb60a92b1222 admin admin] REQUEST_METHOD: `PUT` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:37:41.003798 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1931da6b-bd6d-431b-8876-eb60a92b1222 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:37:41.004076 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1931da6b-bd6d-431b-8876-eb60a92b1222 admin admin] PATH_INFO: `/v3/projects/90377491529542949c3efb5b6475501b/users/7a0895dd27ef4292a312406da3aa441a/roles/8051174e7a7947d5b991ebd793c2e543` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:37:41.020796 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1931da6b-bd6d-431b-8876-eb60a92b1222 admin admin] RBAC: Authorizing `identity:create_grant(project_id=90377491529542949c3efb5b6475501b, user_id=7a0895dd27ef4292a312406da3aa441a, role_id=8051174e7a7947d5b991ebd793c2e543)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:37:41.023993 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1931da6b-bd6d-431b-8876-eb60a92b1222 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:37:41.038484 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1189/2385] 10.222.0.49 () {60 vars in 1482 bytes} [Wed Jun 15 10:37:40 2022] PUT /identity/v3/projects/90377491529542949c3efb5b6475501b/users/7a0895dd27ef4292a312406da3aa441a/roles/8051174e7a7947d5b991ebd793c2e543 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:37:41.042652 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8adcddf-56b9-4bd8-8831-9617df303dae admin admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:37:41.053451 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8adcddf-56b9-4bd8-8831-9617df303dae admin admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:37:41.064910 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:37:41.116642 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91cd603f-1da5-4843-8518-629603d0fe47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:37:41.117326 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91cd603f-1da5-4843-8518-629603d0fe47 admin admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:37:41.117615 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91cd603f-1da5-4843-8518-629603d0fe47 admin admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:37:41.117895 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91cd603f-1da5-4843-8518-629603d0fe47 admin admin] PATH_INFO: `/v3/roles` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:37:41.118633 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91cd603f-1da5-4843-8518-629603d0fe47 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:37:41.120304 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91cd603f-1da5-4843-8518-629603d0fe47 admin admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:37:41.130581 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1197/2386] 10.222.0.49 () {58 vars in 1121 bytes} [Wed Jun 15 10:37:41 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 15 10:37:41.134696 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1931da6b-bd6d-431b-8876-eb60a92b1222 admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:37:41.140384 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:37:41.147773 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1931da6b-bd6d-431b-8876-eb60a92b1222 admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:37:41.153599 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef6d037-f6f1-47c5-bb9a-49540c7d0424 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:37:41.154493 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef6d037-f6f1-47c5-bb9a-49540c7d0424 admin admin] REQUEST_METHOD: `PUT` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:37:41.154816 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef6d037-f6f1-47c5-bb9a-49540c7d0424 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:37:41.155120 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef6d037-f6f1-47c5-bb9a-49540c7d0424 admin admin] PATH_INFO: `/v3/projects/90377491529542949c3efb5b6475501b/users/7a0895dd27ef4292a312406da3aa441a/roles/8051174e7a7947d5b991ebd793c2e543` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:37:41.157202 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef6d037-f6f1-47c5-bb9a-49540c7d0424 admin admin] RBAC: Authorizing `identity:create_grant(project_id=90377491529542949c3efb5b6475501b, user_id=7a0895dd27ef4292a312406da3aa441a, role_id=8051174e7a7947d5b991ebd793c2e543)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:37:41.160606 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef6d037-f6f1-47c5-bb9a-49540c7d0424 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:37:41.171696 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1190/2387] 10.222.0.49 () {60 vars in 1482 bytes} [Wed Jun 15 10:37:41 2022] PUT /identity/v3/projects/90377491529542949c3efb5b6475501b/users/7a0895dd27ef4292a312406da3aa441a/roles/8051174e7a7947d5b991ebd793c2e543 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 10:37:41.184200 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19ba705-cfaa-4548-95d7-7e0cd8e9d9d1 None None] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:37:41.184494 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19ba705-cfaa-4548-95d7-7e0cd8e9d9d1 None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:37:41.184773 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19ba705-cfaa-4548-95d7-7e0cd8e9d9d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:37:41.211016 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.auth.core [None req-f19ba705-cfaa-4548-95d7-7e0cd8e9d9d1 None None] MFA Rules not processed for user `7a0895dd27ef4292a312406da3aa441a`. Rule list: `[]` (Enabled: `True`). {{(pid=80119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:37:41.212358 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-f19ba705-cfaa-4548-95d7-7e0cd8e9d9d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:37:41.280254 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1198/2388] 10.222.0.49 () {58 vars in 957 bytes} [Wed Jun 15 10:37:41 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:37:41.291468 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef6d037-f6f1-47c5-bb9a-49540c7d0424 admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:37:41.293202 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-eef6d037-f6f1-47c5-bb9a-49540c7d0424 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:37:41.349125 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef6d037-f6f1-47c5-bb9a-49540c7d0424 admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:37:41.352675 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc98a2b8-984e-4761-96fa-6516178d8c7f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20d1327553404b80aecee87c78fcb52b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '559b3e713f5844b6ae37151d56dd419f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:37:41.353306 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc98a2b8-984e-4761-96fa-6516178d8c7f service cinder] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:37:41.353589 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc98a2b8-984e-4761-96fa-6516178d8c7f service cinder] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:37:41.353867 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc98a2b8-984e-4761-96fa-6516178d8c7f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:37:41.362756 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc98a2b8-984e-4761-96fa-6516178d8c7f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:37:41.364215 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc98a2b8-984e-4761-96fa-6516178d8c7f service cinder] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:37:41.373559 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1191/2389] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Jun 15 10:37:41 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:37:41.448586 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19ba705-cfaa-4548-95d7-7e0cd8e9d9d1 None None] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:37:41.458934 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19ba705-cfaa-4548-95d7-7e0cd8e9d9d1 None None] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:37:41.462563 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e283902a-07d8-4b60-9a28-9e468482cbc7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20d1327553404b80aecee87c78fcb52b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '559b3e713f5844b6ae37151d56dd419f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:37:41.463274 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e283902a-07d8-4b60-9a28-9e468482cbc7 service cinder] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:37:41.463556 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e283902a-07d8-4b60-9a28-9e468482cbc7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:37:41.463832 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e283902a-07d8-4b60-9a28-9e468482cbc7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:37:41.527229 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e283902a-07d8-4b60-9a28-9e468482cbc7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:37:41.528836 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e283902a-07d8-4b60-9a28-9e468482cbc7 service cinder] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:37:41.538625 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1199/2390] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Jun 15 10:37:41 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:37:43.016983 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5428839d-ed76-4484-93a8-19155f5fbc3f None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:37:43.017523 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5428839d-ed76-4484-93a8-19155f5fbc3f None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:37:43.017523 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5428839d-ed76-4484-93a8-19155f5fbc3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:37:43.046019 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-5428839d-ed76-4484-93a8-19155f5fbc3f None None] MFA Rules not processed for user `103e10244eca444b86b9c51ff314d445`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:37:43.047860 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-5428839d-ed76-4484-93a8-19155f5fbc3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:37:43.117272 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1192/2391] 10.222.0.49 () {58 vars in 967 bytes} [Wed Jun 15 10:37:43 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 10:37:43.132284 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e283902a-07d8-4b60-9a28-9e468482cbc7 service cinder] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:37:43.133742 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-e283902a-07d8-4b60-9a28-9e468482cbc7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:37:43.201302 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=100 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:37:43.237708 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e283902a-07d8-4b60-9a28-9e468482cbc7 service cinder] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:37:43.244200 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a403a848-be32-44a2-9a10-fd5541081f8a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a69f99210444f9696e8b22615fb4a41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '559b3e713f5844b6ae37151d56dd419f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:37:43.244873 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a403a848-be32-44a2-9a10-fd5541081f8a service swift] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:37:43.245171 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a403a848-be32-44a2-9a10-fd5541081f8a service swift] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:37:43.245444 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a403a848-be32-44a2-9a10-fd5541081f8a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:37:43.254717 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a403a848-be32-44a2-9a10-fd5541081f8a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:37:43.256341 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a403a848-be32-44a2-9a10-fd5541081f8a service swift] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:37:43.264674 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1200/2392] 10.222.0.49 () {60 vars in 1384 bytes} [Wed Jun 15 10:37:43 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jun 15 10:37:45.678419 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5428839d-ed76-4484-93a8-19155f5fbc3f None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:37:45.740223 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5428839d-ed76-4484-93a8-19155f5fbc3f None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:37:45.744697 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd36357-ba81-42a0-b73a-284ddce8e0e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:37:45.745558 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd36357-ba81-42a0-b73a-284ddce8e0e9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:37:45.745840 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd36357-ba81-42a0-b73a-284ddce8e0e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:37:45.746140 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd36357-ba81-42a0-b73a-284ddce8e0e9 admin admin] PATH_INFO: `/v3/users/c861f650bd0b4df897d8bd9389ad967c` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:37:45.747337 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd36357-ba81-42a0-b73a-284ddce8e0e9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c861f650bd0b4df897d8bd9389ad967c)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:37:45.749234 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd36357-ba81-42a0-b73a-284ddce8e0e9 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:37:45.789956 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=94,keystone:INSERT=1,keystone:DELETE=5 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:37:45.796984 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-0fd36357-ba81-42a0-b73a-284ddce8e0e9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c861f650bd0b4df897d8bd9389ad967c'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:37:45.805634 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-0fd36357-ba81-42a0-b73a-284ddce8e0e9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c861f650bd0b4df897d8bd9389ad967c'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:37:45.806379 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-0fd36357-ba81-42a0-b73a-284ddce8e0e9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c861f650bd0b4df897d8bd9389ad967c'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:37:45.807030 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-0fd36357-ba81-42a0-b73a-284ddce8e0e9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c861f650bd0b4df897d8bd9389ad967c'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:37:45.814161 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-0fd36357-ba81-42a0-b73a-284ddce8e0e9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c861f650bd0b4df897d8bd9389ad967c'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:37:45.831666 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1193/2393] 10.222.0.49 () {58 vars in 1223 bytes} [Wed Jun 15 10:37:45 2022] DELETE /identity/v3/users/c861f650bd0b4df897d8bd9389ad967c => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:37:45.836098 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61aed91-5ba9-41d1-8707-ece7f2390145 None None] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:37:45.836571 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61aed91-5ba9-41d1-8707-ece7f2390145 None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:37:45.836669 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61aed91-5ba9-41d1-8707-ece7f2390145 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:37:45.853361 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.auth.core [None req-f61aed91-5ba9-41d1-8707-ece7f2390145 None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:37:45.854553 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-f61aed91-5ba9-41d1-8707-ece7f2390145 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:37:45.920261 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1201/2394] 10.222.0.49 () {58 vars in 957 bytes} [Wed Jun 15 10:37:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:37:45.932543 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd36357-ba81-42a0-b73a-284ddce8e0e9 admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:37:45.932543 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-0fd36357-ba81-42a0-b73a-284ddce8e0e9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:37:45.994678 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd36357-ba81-42a0-b73a-284ddce8e0e9 admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:37:45.998328 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35df559b-747b-42fd-b566-d7625eba3817 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0cb44a17d205431793e9a8791e477a32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '559b3e713f5844b6ae37151d56dd419f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:37:45.998983 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35df559b-747b-42fd-b566-d7625eba3817 service neutron] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:37:45.999306 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35df559b-747b-42fd-b566-d7625eba3817 service neutron] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:37:45.999591 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35df559b-747b-42fd-b566-d7625eba3817 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:37:46.008966 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35df559b-747b-42fd-b566-d7625eba3817 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:37:46.010410 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35df559b-747b-42fd-b566-d7625eba3817 service neutron] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:37:46.020745 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1194/2395] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Jun 15 10:37:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 10:37:46.287930 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61aed91-5ba9-41d1-8707-ece7f2390145 None None] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:37:46.289419 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-f61aed91-5ba9-41d1-8707-ece7f2390145 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:37:46.305013 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61aed91-5ba9-41d1-8707-ece7f2390145 None None] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:37:46.310085 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8faa2d8b-5ae3-440c-beee-4cd3b1751b6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:37:46.310953 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8faa2d8b-5ae3-440c-beee-4cd3b1751b6c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:37:46.311266 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8faa2d8b-5ae3-440c-beee-4cd3b1751b6c admin admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:37:46.311544 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8faa2d8b-5ae3-440c-beee-4cd3b1751b6c admin admin] PATH_INFO: `/v3/projects/e7159500a9864347a625ca47a937416f` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:37:46.312722 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8faa2d8b-5ae3-440c-beee-4cd3b1751b6c admin admin] RBAC: Authorizing `identity:delete_project(project_id=e7159500a9864347a625ca47a937416f)` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:37:46.314760 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8faa2d8b-5ae3-440c-beee-4cd3b1751b6c admin admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:37:46.366986 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-8faa2d8b-5ae3-440c-beee-4cd3b1751b6c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e7159500a9864347a625ca47a937416f', 'request_id': 'req-8faa2d8b-5ae3-440c-beee-4cd3b1751b6c'} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:37:46.375386 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-8faa2d8b-5ae3-440c-beee-4cd3b1751b6c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e7159500a9864347a625ca47a937416f', 'request_id': 'req-8faa2d8b-5ae3-440c-beee-4cd3b1751b6c'} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:37:46.381283 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-8faa2d8b-5ae3-440c-beee-4cd3b1751b6c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e7159500a9864347a625ca47a937416f', 'request_id': 'req-8faa2d8b-5ae3-440c-beee-4cd3b1751b6c'} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:37:46.393562 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-8faa2d8b-5ae3-440c-beee-4cd3b1751b6c admin admin] The token cache is being invalidate because project e7159500a9864347a625ca47a937416f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80119) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 10:37:46.393898 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-8faa2d8b-5ae3-440c-beee-4cd3b1751b6c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:37:46.394542 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-8faa2d8b-5ae3-440c-beee-4cd3b1751b6c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:37:46.396115 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1202/2396] 10.222.0.49 () {58 vars in 1232 bytes} [Wed Jun 15 10:37:46 2022] DELETE /identity/v3/projects/e7159500a9864347a625ca47a937416f => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:37:46.400080 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35df559b-747b-42fd-b566-d7625eba3817 service neutron] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:37:46.410690 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35df559b-747b-42fd-b566-d7625eba3817 service neutron] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:37:46.465116 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d8e508-b18d-4c7a-8905-d06cb6b8d0d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:37:46.465983 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d8e508-b18d-4c7a-8905-d06cb6b8d0d6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:37:46.466266 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d8e508-b18d-4c7a-8905-d06cb6b8d0d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:37:46.466541 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d8e508-b18d-4c7a-8905-d06cb6b8d0d6 admin admin] PATH_INFO: `/v3/users/7a0895dd27ef4292a312406da3aa441a` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:37:46.467743 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d8e508-b18d-4c7a-8905-d06cb6b8d0d6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7a0895dd27ef4292a312406da3aa441a)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:37:46.469657 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d8e508-b18d-4c7a-8905-d06cb6b8d0d6 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:37:46.512902 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-19d8e508-b18d-4c7a-8905-d06cb6b8d0d6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7a0895dd27ef4292a312406da3aa441a'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:37:46.519990 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-19d8e508-b18d-4c7a-8905-d06cb6b8d0d6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7a0895dd27ef4292a312406da3aa441a'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:37:46.520637 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-19d8e508-b18d-4c7a-8905-d06cb6b8d0d6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7a0895dd27ef4292a312406da3aa441a'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:37:46.521240 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-19d8e508-b18d-4c7a-8905-d06cb6b8d0d6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7a0895dd27ef4292a312406da3aa441a'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:37:46.524876 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:DELETE=13,keystone:SELECT=85,keystone:INSERT=2 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:37:46.528438 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-19d8e508-b18d-4c7a-8905-d06cb6b8d0d6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7a0895dd27ef4292a312406da3aa441a'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:37:46.548116 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1195/2397] 10.222.0.49 () {58 vars in 1223 bytes} [Wed Jun 15 10:37:46 2022] DELETE /identity/v3/users/7a0895dd27ef4292a312406da3aa441a => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:37:46.819257 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8faa2d8b-5ae3-440c-beee-4cd3b1751b6c admin admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:37:46.820698 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-8faa2d8b-5ae3-440c-beee-4cd3b1751b6c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:37:46.835584 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8faa2d8b-5ae3-440c-beee-4cd3b1751b6c admin admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:37:46.883519 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:37:46.897887 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7c92aa-e48c-42c4-9ee1-fbeb4ac6903e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:37:46.898742 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7c92aa-e48c-42c4-9ee1-fbeb4ac6903e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:37:46.899022 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7c92aa-e48c-42c4-9ee1-fbeb4ac6903e admin admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:37:46.899341 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7c92aa-e48c-42c4-9ee1-fbeb4ac6903e admin admin] PATH_INFO: `/v3/projects/90377491529542949c3efb5b6475501b` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:37:46.900754 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c7c92aa-e48c-42c4-9ee1-fbeb4ac6903e admin admin] RBAC: Authorizing `identity:delete_project(project_id=90377491529542949c3efb5b6475501b)` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:37:46.902652 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c7c92aa-e48c-42c4-9ee1-fbeb4ac6903e admin admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:37:46.947884 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-4c7c92aa-e48c-42c4-9ee1-fbeb4ac6903e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '90377491529542949c3efb5b6475501b', 'request_id': 'req-4c7c92aa-e48c-42c4-9ee1-fbeb4ac6903e'} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:37:46.954677 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-4c7c92aa-e48c-42c4-9ee1-fbeb4ac6903e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '90377491529542949c3efb5b6475501b', 'request_id': 'req-4c7c92aa-e48c-42c4-9ee1-fbeb4ac6903e'} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:37:46.960795 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-4c7c92aa-e48c-42c4-9ee1-fbeb4ac6903e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '90377491529542949c3efb5b6475501b', 'request_id': 'req-4c7c92aa-e48c-42c4-9ee1-fbeb4ac6903e'} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:37:46.971372 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-4c7c92aa-e48c-42c4-9ee1-fbeb4ac6903e admin admin] The token cache is being invalidate because project 90377491529542949c3efb5b6475501b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80119) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 10:37:46.971657 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-4c7c92aa-e48c-42c4-9ee1-fbeb4ac6903e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:37:46.972310 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-4c7c92aa-e48c-42c4-9ee1-fbeb4ac6903e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:37:46.973836 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1203/2398] 10.222.0.49 () {58 vars in 1232 bytes} [Wed Jun 15 10:37:46 2022] DELETE /identity/v3/projects/90377491529542949c3efb5b6475501b => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:37:46.978835 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6d34cf-bf97-46a3-a1f8-d043b6141bf3 None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:37:46.979357 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6d34cf-bf97-46a3-a1f8-d043b6141bf3 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:37:46.979587 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6d34cf-bf97-46a3-a1f8-d043b6141bf3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:37:46.996724 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-ad6d34cf-bf97-46a3-a1f8-d043b6141bf3 None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:37:46.998200 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-ad6d34cf-bf97-46a3-a1f8-d043b6141bf3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:37:47.056658 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1196/2399] 10.222.0.49 () {58 vars in 957 bytes} [Wed Jun 15 10:37:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:37:47.066008 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f230b5-0926-4446-b342-20091a0512dc None None] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:37:47.066239 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f230b5-0926-4446-b342-20091a0512dc None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:37:47.066518 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f230b5-0926-4446-b342-20091a0512dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:37:47.085586 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.auth.core [None req-d6f230b5-0926-4446-b342-20091a0512dc None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:37:47.086785 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-d6f230b5-0926-4446-b342-20091a0512dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:37:47.095236 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1204/2400] 10.222.0.49 () {58 vars in 957 bytes} [Wed Jun 15 10:37:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:37:47.099795 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6d34cf-bf97-46a3-a1f8-d043b6141bf3 None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:37:47.109693 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6d34cf-bf97-46a3-a1f8-d043b6141bf3 None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:37:47.114831 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c71256-7de4-4db0-bd2c-8b4a4e6de8df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:37:47.115519 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c71256-7de4-4db0-bd2c-8b4a4e6de8df admin admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:37:47.115807 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c71256-7de4-4db0-bd2c-8b4a4e6de8df admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:37:47.116086 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c71256-7de4-4db0-bd2c-8b4a4e6de8df admin admin] PATH_INFO: `/v3/domains` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:37:47.116807 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c71256-7de4-4db0-bd2c-8b4a4e6de8df admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:37:47.117109 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c71256-7de4-4db0-bd2c-8b4a4e6de8df admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:37:47.118701 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c71256-7de4-4db0-bd2c-8b4a4e6de8df admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:37:47.138219 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1197/2401] 10.222.0.49 () {58 vars in 1152 bytes} [Wed Jun 15 10:37:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:37:47.139499 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f230b5-0926-4446-b342-20091a0512dc None None] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:37:47.148777 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f230b5-0926-4446-b342-20091a0512dc None None] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:37:47.153671 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1a1646-ea63-44b8-8d0c-39f3678d0a90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:37:47.154324 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1a1646-ea63-44b8-8d0c-39f3678d0a90 admin admin] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:37:47.154605 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1a1646-ea63-44b8-8d0c-39f3678d0a90 admin admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:37:47.154880 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1a1646-ea63-44b8-8d0c-39f3678d0a90 admin admin] PATH_INFO: `/v3/projects` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:37:47.155593 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b1a1646-ea63-44b8-8d0c-39f3678d0a90 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:37:47.157494 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b1a1646-ea63-44b8-8d0c-39f3678d0a90 admin admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:37:47.187531 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1205/2402] 10.222.0.49 () {60 vars in 1152 bytes} [Wed Jun 15 10:37:47 2022] POST /identity/v3/projects => generated 377 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 10:37:47.192358 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c71256-7de4-4db0-bd2c-8b4a4e6de8df admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:37:47.207739 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c71256-7de4-4db0-bd2c-8b4a4e6de8df admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:37:47.269323 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a862e736-340b-4036-9963-23004a6ad33b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:37:47.270128 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a862e736-340b-4036-9963-23004a6ad33b admin admin] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:37:47.270445 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a862e736-340b-4036-9963-23004a6ad33b admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:37:47.270768 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a862e736-340b-4036-9963-23004a6ad33b admin admin] PATH_INFO: `/v3/users` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:37:47.271575 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a862e736-340b-4036-9963-23004a6ad33b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:37:47.273715 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a862e736-340b-4036-9963-23004a6ad33b admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:37:47.298841 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1198/2403] 10.222.0.49 () {60 vars in 1143 bytes} [Wed Jun 15 10:37:47 2022] POST /identity/v3/users => generated 289 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 10:37:47.309432 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1a1646-ea63-44b8-8d0c-39f3678d0a90 admin admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:37:47.321178 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1a1646-ea63-44b8-8d0c-39f3678d0a90 admin admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:37:47.325987 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926de1c9-b253-4329-b7a8-bce79b7a4490 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:37:47.326657 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926de1c9-b253-4329-b7a8-bce79b7a4490 admin admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:37:47.326944 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926de1c9-b253-4329-b7a8-bce79b7a4490 admin admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:37:47.327382 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926de1c9-b253-4329-b7a8-bce79b7a4490 admin admin] PATH_INFO: `/v3/roles` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:37:47.328116 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-926de1c9-b253-4329-b7a8-bce79b7a4490 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:37:47.329720 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-926de1c9-b253-4329-b7a8-bce79b7a4490 admin admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:37:47.344274 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1206/2404] 10.222.0.49 () {58 vars in 1121 bytes} [Wed Jun 15 10:37:47 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 10:37:47.348173 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a862e736-340b-4036-9963-23004a6ad33b admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:37:47.357956 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a862e736-340b-4036-9963-23004a6ad33b admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:37:47.363077 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80969c6f-0d41-44e1-a4c0-0d7886e8be53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:37:47.363764 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80969c6f-0d41-44e1-a4c0-0d7886e8be53 admin admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:37:47.364050 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80969c6f-0d41-44e1-a4c0-0d7886e8be53 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:37:47.364328 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80969c6f-0d41-44e1-a4c0-0d7886e8be53 admin admin] PATH_INFO: `/v3/roles` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:37:47.365050 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80969c6f-0d41-44e1-a4c0-0d7886e8be53 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:37:47.366673 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80969c6f-0d41-44e1-a4c0-0d7886e8be53 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:37:47.376521 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1199/2405] 10.222.0.49 () {58 vars in 1121 bytes} [Wed Jun 15 10:37:47 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 15 10:37:47.381492 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926de1c9-b253-4329-b7a8-bce79b7a4490 admin admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:37:47.391734 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926de1c9-b253-4329-b7a8-bce79b7a4490 admin admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:37:47.396248 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e41c531-dff6-4234-a562-bb0779bea943 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:37:47.397121 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e41c531-dff6-4234-a562-bb0779bea943 admin admin] REQUEST_METHOD: `PUT` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:37:47.397402 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e41c531-dff6-4234-a562-bb0779bea943 admin admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:37:47.397679 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e41c531-dff6-4234-a562-bb0779bea943 admin admin] PATH_INFO: `/v3/projects/369db092ec72463b9ff27caed4e828f8/users/b0aad8a3cc054e2f98ed82488fe0a034/roles/6876e57cc51c448097ab109d35b2a3da` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:37:47.412109 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e41c531-dff6-4234-a562-bb0779bea943 admin admin] RBAC: Authorizing `identity:create_grant(project_id=369db092ec72463b9ff27caed4e828f8, user_id=b0aad8a3cc054e2f98ed82488fe0a034, role_id=6876e57cc51c448097ab109d35b2a3da)` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:37:47.414737 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e41c531-dff6-4234-a562-bb0779bea943 admin admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:37:47.427482 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1207/2406] 10.222.0.49 () {60 vars in 1482 bytes} [Wed Jun 15 10:37:47 2022] PUT /identity/v3/projects/369db092ec72463b9ff27caed4e828f8/users/b0aad8a3cc054e2f98ed82488fe0a034/roles/6876e57cc51c448097ab109d35b2a3da => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:37:47.437522 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29668fa-8f77-4d31-ab7d-2f35de246be0 None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:37:47.437813 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29668fa-8f77-4d31-ab7d-2f35de246be0 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:37:47.438089 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29668fa-8f77-4d31-ab7d-2f35de246be0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:37:47.465734 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-a29668fa-8f77-4d31-ab7d-2f35de246be0 None None] MFA Rules not processed for user `b0aad8a3cc054e2f98ed82488fe0a034`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:37:47.466948 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-a29668fa-8f77-4d31-ab7d-2f35de246be0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:37:47.500564 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:37:47.535470 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1200/2407] 10.222.0.49 () {58 vars in 957 bytes} [Wed Jun 15 10:37:47 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:37:47.539995 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e41c531-dff6-4234-a562-bb0779bea943 admin admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:37:47.603938 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e41c531-dff6-4234-a562-bb0779bea943 admin admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:37:47.609489 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d893a09e-a018-4958-8cdf-41391aa538e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:37:47.610344 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d893a09e-a018-4958-8cdf-41391aa538e3 admin admin] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:37:47.610565 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d893a09e-a018-4958-8cdf-41391aa538e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:37:47.610870 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d893a09e-a018-4958-8cdf-41391aa538e3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:37:47.612335 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d893a09e-a018-4958-8cdf-41391aa538e3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:37:47.614263 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d893a09e-a018-4958-8cdf-41391aa538e3 admin admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:37:47.633604 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:37:47.640997 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1208/2408] 10.222.0.49 () {60 vars in 1152 bytes} [Wed Jun 15 10:37:47 2022] POST /identity/v3/projects => generated 377 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 10:37:47.645046 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29668fa-8f77-4d31-ab7d-2f35de246be0 None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:37:47.655984 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29668fa-8f77-4d31-ab7d-2f35de246be0 None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:37:47.712102 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1282b1f-8096-4840-8d9e-66536665cfdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:37:47.712764 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1282b1f-8096-4840-8d9e-66536665cfdc admin admin] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:37:47.713045 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1282b1f-8096-4840-8d9e-66536665cfdc admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:37:47.713320 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1282b1f-8096-4840-8d9e-66536665cfdc admin admin] PATH_INFO: `/v3/users` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:37:47.714008 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1282b1f-8096-4840-8d9e-66536665cfdc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:37:47.715950 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1282b1f-8096-4840-8d9e-66536665cfdc admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:37:47.736425 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1201/2409] 10.222.0.49 () {60 vars in 1143 bytes} [Wed Jun 15 10:37:47 2022] POST /identity/v3/users => generated 289 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 10:37:47.740952 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d893a09e-a018-4958-8cdf-41391aa538e3 admin admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:37:47.751225 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d893a09e-a018-4958-8cdf-41391aa538e3 admin admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:37:47.755912 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7f1e55-fe92-4f7a-a639-997eccd6bf4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:37:47.756570 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7f1e55-fe92-4f7a-a639-997eccd6bf4f admin admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:37:47.756858 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7f1e55-fe92-4f7a-a639-997eccd6bf4f admin admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:37:47.757135 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7f1e55-fe92-4f7a-a639-997eccd6bf4f admin admin] PATH_INFO: `/v3/roles` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:37:47.757853 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff7f1e55-fe92-4f7a-a639-997eccd6bf4f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:37:47.759532 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff7f1e55-fe92-4f7a-a639-997eccd6bf4f admin admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:37:47.770601 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1209/2410] 10.222.0.49 () {58 vars in 1121 bytes} [Wed Jun 15 10:37:47 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 10:37:47.780110 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1282b1f-8096-4840-8d9e-66536665cfdc admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:37:47.796203 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1282b1f-8096-4840-8d9e-66536665cfdc admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:37:47.801556 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a55d8e-0f58-414a-8e2f-667bbf94bf11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:37:47.802232 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a55d8e-0f58-414a-8e2f-667bbf94bf11 admin admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:37:47.802521 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a55d8e-0f58-414a-8e2f-667bbf94bf11 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:37:47.802802 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a55d8e-0f58-414a-8e2f-667bbf94bf11 admin admin] PATH_INFO: `/v3/roles` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:37:47.803573 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a55d8e-0f58-414a-8e2f-667bbf94bf11 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:37:47.805177 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a55d8e-0f58-414a-8e2f-667bbf94bf11 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:37:47.815370 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1202/2411] 10.222.0.49 () {58 vars in 1121 bytes} [Wed Jun 15 10:37:47 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 10:37:47.819502 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7f1e55-fe92-4f7a-a639-997eccd6bf4f admin admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:37:47.829356 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7f1e55-fe92-4f7a-a639-997eccd6bf4f admin admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:37:47.834541 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8f15ac-f410-4e55-8bc0-82357ebb88eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:37:47.835462 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8f15ac-f410-4e55-8bc0-82357ebb88eb admin admin] REQUEST_METHOD: `PUT` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:37:47.835746 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8f15ac-f410-4e55-8bc0-82357ebb88eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:37:47.836023 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8f15ac-f410-4e55-8bc0-82357ebb88eb admin admin] PATH_INFO: `/v3/projects/f287efd29e8e411291b802cc5181fa5d/users/349b4db3fc1a40b487a87f58034f7013/roles/6876e57cc51c448097ab109d35b2a3da` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:37:47.851012 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8f15ac-f410-4e55-8bc0-82357ebb88eb admin admin] RBAC: Authorizing `identity:create_grant(project_id=f287efd29e8e411291b802cc5181fa5d, user_id=349b4db3fc1a40b487a87f58034f7013, role_id=6876e57cc51c448097ab109d35b2a3da)` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:37:47.853622 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8f15ac-f410-4e55-8bc0-82357ebb88eb admin admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:37:47.866140 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1210/2412] 10.222.0.49 () {60 vars in 1482 bytes} [Wed Jun 15 10:37:47 2022] PUT /identity/v3/projects/f287efd29e8e411291b802cc5181fa5d/users/349b4db3fc1a40b487a87f58034f7013/roles/6876e57cc51c448097ab109d35b2a3da => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:37:47.876340 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710bafd4-3d86-491a-b0d6-5e8c5239a8ef None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:37:47.876623 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710bafd4-3d86-491a-b0d6-5e8c5239a8ef None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:37:47.876899 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710bafd4-3d86-491a-b0d6-5e8c5239a8ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:37:47.902999 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-710bafd4-3d86-491a-b0d6-5e8c5239a8ef None None] MFA Rules not processed for user `349b4db3fc1a40b487a87f58034f7013`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:37:47.904228 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-710bafd4-3d86-491a-b0d6-5e8c5239a8ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:37:47.966607 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1203/2413] 10.222.0.49 () {58 vars in 957 bytes} [Wed Jun 15 10:37:47 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:37:47.972551 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8f15ac-f410-4e55-8bc0-82357ebb88eb admin admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:37:48.034411 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8f15ac-f410-4e55-8bc0-82357ebb88eb admin admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:37:48.039114 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361387c5-70e4-4058-8780-8a3b52c68bc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:37:48.039786 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361387c5-70e4-4058-8780-8a3b52c68bc6 admin admin] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:37:48.040074 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361387c5-70e4-4058-8780-8a3b52c68bc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:37:48.040361 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361387c5-70e4-4058-8780-8a3b52c68bc6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:37:48.041046 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-361387c5-70e4-4058-8780-8a3b52c68bc6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:37:48.042952 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-361387c5-70e4-4058-8780-8a3b52c68bc6 admin admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:37:48.067305 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1211/2414] 10.222.0.49 () {60 vars in 1152 bytes} [Wed Jun 15 10:37:47 2022] POST /identity/v3/projects => generated 377 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 10:37:48.071330 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710bafd4-3d86-491a-b0d6-5e8c5239a8ef None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:37:48.082515 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710bafd4-3d86-491a-b0d6-5e8c5239a8ef None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:37:48.121705 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:37:48.138603 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c7d33c-0aaa-41fe-bb96-76c841706fb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:37:48.139302 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c7d33c-0aaa-41fe-bb96-76c841706fb9 admin admin] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:37:48.139584 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c7d33c-0aaa-41fe-bb96-76c841706fb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:37:48.139860 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c7d33c-0aaa-41fe-bb96-76c841706fb9 admin admin] PATH_INFO: `/v3/users` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:37:48.140580 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c7d33c-0aaa-41fe-bb96-76c841706fb9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:37:48.142483 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c7d33c-0aaa-41fe-bb96-76c841706fb9 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:37:48.161943 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1204/2415] 10.222.0.49 () {60 vars in 1143 bytes} [Wed Jun 15 10:37:48 2022] POST /identity/v3/users => generated 295 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 10:37:48.165875 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361387c5-70e4-4058-8780-8a3b52c68bc6 admin admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:37:48.176259 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361387c5-70e4-4058-8780-8a3b52c68bc6 admin admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:37:48.180873 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2ea44b-ba57-4312-bd40-e3e1e0e40b77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:37:48.181527 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2ea44b-ba57-4312-bd40-e3e1e0e40b77 admin admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:37:48.181824 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2ea44b-ba57-4312-bd40-e3e1e0e40b77 admin admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:37:48.182086 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2ea44b-ba57-4312-bd40-e3e1e0e40b77 admin admin] PATH_INFO: `/v3/roles` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:37:48.182805 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e2ea44b-ba57-4312-bd40-e3e1e0e40b77 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:37:48.184443 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e2ea44b-ba57-4312-bd40-e3e1e0e40b77 admin admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:37:48.196323 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1212/2416] 10.222.0.49 () {58 vars in 1121 bytes} [Wed Jun 15 10:37:48 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 10:37:48.200373 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c7d33c-0aaa-41fe-bb96-76c841706fb9 admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:37:48.210679 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c7d33c-0aaa-41fe-bb96-76c841706fb9 admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:37:48.215294 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1146eaba-20d2-41a7-a7b7-0e613d3661d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:37:48.216167 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1146eaba-20d2-41a7-a7b7-0e613d3661d2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:37:48.216452 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1146eaba-20d2-41a7-a7b7-0e613d3661d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:37:48.216730 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1146eaba-20d2-41a7-a7b7-0e613d3661d2 admin admin] PATH_INFO: `/v3/projects/4dde1515c2ac4e38a76b81aeaa2347e8/users/f5ab6cdc941f4d8d9081c1733fa12abf/roles/8051174e7a7947d5b991ebd793c2e543` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:37:48.230975 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1146eaba-20d2-41a7-a7b7-0e613d3661d2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4dde1515c2ac4e38a76b81aeaa2347e8, user_id=f5ab6cdc941f4d8d9081c1733fa12abf, role_id=8051174e7a7947d5b991ebd793c2e543)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:37:48.233643 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1146eaba-20d2-41a7-a7b7-0e613d3661d2 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:37:48.246629 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1205/2417] 10.222.0.49 () {60 vars in 1482 bytes} [Wed Jun 15 10:37:48 2022] PUT /identity/v3/projects/4dde1515c2ac4e38a76b81aeaa2347e8/users/f5ab6cdc941f4d8d9081c1733fa12abf/roles/8051174e7a7947d5b991ebd793c2e543 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:37:48.250499 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2ea44b-ba57-4312-bd40-e3e1e0e40b77 admin admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:37:48.260855 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2ea44b-ba57-4312-bd40-e3e1e0e40b77 admin admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:37:48.317451 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9332e1-d053-4899-8c0c-fc564c04068a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:37:48.318121 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9332e1-d053-4899-8c0c-fc564c04068a admin admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:37:48.318405 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9332e1-d053-4899-8c0c-fc564c04068a admin admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:37:48.318683 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9332e1-d053-4899-8c0c-fc564c04068a admin admin] PATH_INFO: `/v3/roles` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:37:48.319457 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9332e1-d053-4899-8c0c-fc564c04068a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:37:48.321055 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9332e1-d053-4899-8c0c-fc564c04068a admin admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:37:48.331249 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1213/2418] 10.222.0.49 () {58 vars in 1121 bytes} [Wed Jun 15 10:37:48 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 10:37:48.335285 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1146eaba-20d2-41a7-a7b7-0e613d3661d2 admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:37:48.345725 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1146eaba-20d2-41a7-a7b7-0e613d3661d2 admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:37:48.350535 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb83403-9b39-432b-bebe-5a4861f2a3a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:37:48.351480 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb83403-9b39-432b-bebe-5a4861f2a3a1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:37:48.351760 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb83403-9b39-432b-bebe-5a4861f2a3a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:37:48.352036 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb83403-9b39-432b-bebe-5a4861f2a3a1 admin admin] PATH_INFO: `/v3/projects/4dde1515c2ac4e38a76b81aeaa2347e8/users/f5ab6cdc941f4d8d9081c1733fa12abf/roles/8051174e7a7947d5b991ebd793c2e543` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:37:48.354011 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb83403-9b39-432b-bebe-5a4861f2a3a1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4dde1515c2ac4e38a76b81aeaa2347e8, user_id=f5ab6cdc941f4d8d9081c1733fa12abf, role_id=8051174e7a7947d5b991ebd793c2e543)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:37:48.356657 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb83403-9b39-432b-bebe-5a4861f2a3a1 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:37:48.370766 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1206/2419] 10.222.0.49 () {60 vars in 1482 bytes} [Wed Jun 15 10:37:48 2022] PUT /identity/v3/projects/4dde1515c2ac4e38a76b81aeaa2347e8/users/f5ab6cdc941f4d8d9081c1733fa12abf/roles/8051174e7a7947d5b991ebd793c2e543 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:37:48.382327 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d7585f-da7d-4830-8f16-c0c533441210 None None] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:37:48.382618 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d7585f-da7d-4830-8f16-c0c533441210 None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:37:48.382897 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d7585f-da7d-4830-8f16-c0c533441210 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:37:48.419996 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.auth.core [None req-d6d7585f-da7d-4830-8f16-c0c533441210 None None] MFA Rules not processed for user `f5ab6cdc941f4d8d9081c1733fa12abf`. Rule list: `[]` (Enabled: `True`). {{(pid=80119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:37:48.421226 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-d6d7585f-da7d-4830-8f16-c0c533441210 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:37:48.444451 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb83403-9b39-432b-bebe-5a4861f2a3a1 admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:37:48.446466 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-1fb83403-9b39-432b-bebe-5a4861f2a3a1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:37:48.448731 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:37:48.495449 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1214/2420] 10.222.0.49 () {58 vars in 957 bytes} [Wed Jun 15 10:37:48 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:37:48.505377 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb83403-9b39-432b-bebe-5a4861f2a3a1 admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:37:48.507382 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d7585f-da7d-4830-8f16-c0c533441210 None None] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:37:48.508725 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-d6d7585f-da7d-4830-8f16-c0c533441210 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:37:48.509136 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1acd4561-eae6-4de5-9b8c-898d281a8d4f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0cb44a17d205431793e9a8791e477a32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '559b3e713f5844b6ae37151d56dd419f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:37:48.509859 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acd4561-eae6-4de5-9b8c-898d281a8d4f service neutron] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:37:48.510137 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acd4561-eae6-4de5-9b8c-898d281a8d4f service neutron] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:37:48.510412 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acd4561-eae6-4de5-9b8c-898d281a8d4f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:37:48.512217 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-1acd4561-eae6-4de5-9b8c-898d281a8d4f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:37:48.565821 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d7585f-da7d-4830-8f16-c0c533441210 None None] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:37:48.569540 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4800ec00-094e-4b1a-a88d-f49391164b3c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20d1327553404b80aecee87c78fcb52b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '559b3e713f5844b6ae37151d56dd419f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:37:48.570188 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4800ec00-094e-4b1a-a88d-f49391164b3c service cinder] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:37:48.570476 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4800ec00-094e-4b1a-a88d-f49391164b3c service cinder] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:37:48.570758 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4800ec00-094e-4b1a-a88d-f49391164b3c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:37:48.575602 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1acd4561-eae6-4de5-9b8c-898d281a8d4f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:37:48.577031 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1acd4561-eae6-4de5-9b8c-898d281a8d4f service neutron] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:37:48.587291 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1207/2421] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Jun 15 10:37:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 10:37:48.626888 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4800ec00-094e-4b1a-a88d-f49391164b3c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:37:48.629172 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4800ec00-094e-4b1a-a88d-f49391164b3c service cinder] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:37:48.639281 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1215/2422] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Jun 15 10:37:48 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 10:37:50.142210 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1acd4561-eae6-4de5-9b8c-898d281a8d4f service neutron] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:37:50.143878 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-1acd4561-eae6-4de5-9b8c-898d281a8d4f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:37:50.156023 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1acd4561-eae6-4de5-9b8c-898d281a8d4f service neutron] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:37:50.159868 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dbbdebe-3500-40d3-aac3-4de8c38764ba service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20d1327553404b80aecee87c78fcb52b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '559b3e713f5844b6ae37151d56dd419f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:37:50.160522 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbbdebe-3500-40d3-aac3-4de8c38764ba service cinder] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:37:50.160808 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbbdebe-3500-40d3-aac3-4de8c38764ba service cinder] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:37:50.161087 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbbdebe-3500-40d3-aac3-4de8c38764ba service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:37:50.210570 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:37:50.227873 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dbbdebe-3500-40d3-aac3-4de8c38764ba service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:37:50.229592 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dbbdebe-3500-40d3-aac3-4de8c38764ba service cinder] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:37:50.239797 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1208/2423] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Jun 15 10:37:50 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:37:50.829433 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4800ec00-094e-4b1a-a88d-f49391164b3c service cinder] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:37:50.838869 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4800ec00-094e-4b1a-a88d-f49391164b3c service cinder] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:37:50.842520 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e65cd2-71e1-47d8-b6db-a1322362701b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20d1327553404b80aecee87c78fcb52b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '559b3e713f5844b6ae37151d56dd419f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:37:50.843201 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e65cd2-71e1-47d8-b6db-a1322362701b service cinder] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:37:50.843494 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e65cd2-71e1-47d8-b6db-a1322362701b service cinder] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:37:50.843770 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e65cd2-71e1-47d8-b6db-a1322362701b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:37:50.853412 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e65cd2-71e1-47d8-b6db-a1322362701b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:37:50.854852 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e65cd2-71e1-47d8-b6db-a1322362701b service cinder] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:37:50.864603 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1216/2424] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Jun 15 10:37:50 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:37:51.787655 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dbbdebe-3500-40d3-aac3-4de8c38764ba service cinder] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:37:51.789710 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-7dbbdebe-3500-40d3-aac3-4de8c38764ba service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:37:51.805678 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dbbdebe-3500-40d3-aac3-4de8c38764ba service cinder] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:37:51.811930 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e03551-0ad4-45b9-8c6b-aa97e75a866d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:37:51.812822 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e03551-0ad4-45b9-8c6b-aa97e75a866d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:37:51.813112 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e03551-0ad4-45b9-8c6b-aa97e75a866d admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:37:51.813392 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e03551-0ad4-45b9-8c6b-aa97e75a866d admin admin] PATH_INFO: `/v3/users/14b720a2d0854d218272390205887100` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:37:51.814629 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e03551-0ad4-45b9-8c6b-aa97e75a866d admin admin] RBAC: Authorizing `identity:delete_user(user_id=14b720a2d0854d218272390205887100)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:37:51.816594 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e03551-0ad4-45b9-8c6b-aa97e75a866d admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:37:51.863571 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-42e03551-0ad4-45b9-8c6b-aa97e75a866d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '14b720a2d0854d218272390205887100'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:37:51.870445 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-42e03551-0ad4-45b9-8c6b-aa97e75a866d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '14b720a2d0854d218272390205887100'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:37:51.871142 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-42e03551-0ad4-45b9-8c6b-aa97e75a866d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '14b720a2d0854d218272390205887100'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:37:51.871761 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-42e03551-0ad4-45b9-8c6b-aa97e75a866d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '14b720a2d0854d218272390205887100'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:37:51.878988 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-42e03551-0ad4-45b9-8c6b-aa97e75a866d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '14b720a2d0854d218272390205887100'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:37:51.897771 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1209/2425] 10.222.0.49 () {58 vars in 1223 bytes} [Wed Jun 15 10:37:51 2022] DELETE /identity/v3/users/14b720a2d0854d218272390205887100 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:37:52.076575 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e65cd2-71e1-47d8-b6db-a1322362701b service cinder] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:37:52.091854 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e65cd2-71e1-47d8-b6db-a1322362701b service cinder] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:37:52.153884 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4df645d-c7a9-4be8-925b-a19d3b13d05a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:37:52.154898 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4df645d-c7a9-4be8-925b-a19d3b13d05a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:37:52.155222 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4df645d-c7a9-4be8-925b-a19d3b13d05a admin admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:37:52.155510 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4df645d-c7a9-4be8-925b-a19d3b13d05a admin admin] PATH_INFO: `/v3/projects/b683b2d399b54484aa5820537038b8af` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:37:52.156667 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4df645d-c7a9-4be8-925b-a19d3b13d05a admin admin] RBAC: Authorizing `identity:delete_project(project_id=b683b2d399b54484aa5820537038b8af)` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:37:52.158636 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4df645d-c7a9-4be8-925b-a19d3b13d05a admin admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:37:52.160979 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=100 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:37:52.224820 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-a4df645d-c7a9-4be8-925b-a19d3b13d05a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b683b2d399b54484aa5820537038b8af', 'request_id': 'req-a4df645d-c7a9-4be8-925b-a19d3b13d05a'} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:37:52.233719 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-a4df645d-c7a9-4be8-925b-a19d3b13d05a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b683b2d399b54484aa5820537038b8af', 'request_id': 'req-a4df645d-c7a9-4be8-925b-a19d3b13d05a'} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:37:52.242255 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-a4df645d-c7a9-4be8-925b-a19d3b13d05a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b683b2d399b54484aa5820537038b8af', 'request_id': 'req-a4df645d-c7a9-4be8-925b-a19d3b13d05a'} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:37:52.256285 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-a4df645d-c7a9-4be8-925b-a19d3b13d05a admin admin] The token cache is being invalidate because project b683b2d399b54484aa5820537038b8af was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80119) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 10:37:52.256702 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-a4df645d-c7a9-4be8-925b-a19d3b13d05a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:37:52.257563 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-a4df645d-c7a9-4be8-925b-a19d3b13d05a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:37:52.259237 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1217/2426] 10.222.0.49 () {58 vars in 1232 bytes} [Wed Jun 15 10:37:52 2022] DELETE /identity/v3/projects/b683b2d399b54484aa5820537038b8af => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:37:52.264100 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b28844-3cd3-45a0-9ec1-4c8a3179f72a None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:37:52.264466 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b28844-3cd3-45a0-9ec1-4c8a3179f72a None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:37:52.264832 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b28844-3cd3-45a0-9ec1-4c8a3179f72a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:37:52.290217 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-26b28844-3cd3-45a0-9ec1-4c8a3179f72a None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:37:52.291568 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-26b28844-3cd3-45a0-9ec1-4c8a3179f72a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:37:52.354443 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1210/2427] 10.222.0.49 () {58 vars in 957 bytes} [Wed Jun 15 10:37:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:37:52.362215 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960e67fa-b9a2-4e5b-9844-b43c912aad1e None None] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:37:52.362510 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960e67fa-b9a2-4e5b-9844-b43c912aad1e None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:37:52.362790 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960e67fa-b9a2-4e5b-9844-b43c912aad1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:37:52.380958 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.auth.core [None req-960e67fa-b9a2-4e5b-9844-b43c912aad1e None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:37:52.382223 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-960e67fa-b9a2-4e5b-9844-b43c912aad1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:37:52.390356 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1218/2428] 10.222.0.49 () {58 vars in 957 bytes} [Wed Jun 15 10:37:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:37:52.394983 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b28844-3cd3-45a0-9ec1-4c8a3179f72a None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:37:52.404931 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b28844-3cd3-45a0-9ec1-4c8a3179f72a None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:37:52.410047 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c46f72-3a93-4e2f-9a1f-d17c443e1674 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:37:52.410701 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c46f72-3a93-4e2f-9a1f-d17c443e1674 admin admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:37:52.410988 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c46f72-3a93-4e2f-9a1f-d17c443e1674 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:37:52.411300 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c46f72-3a93-4e2f-9a1f-d17c443e1674 admin admin] PATH_INFO: `/v3/domains` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:37:52.412030 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c46f72-3a93-4e2f-9a1f-d17c443e1674 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:37:52.412340 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c46f72-3a93-4e2f-9a1f-d17c443e1674 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:37:52.413933 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c46f72-3a93-4e2f-9a1f-d17c443e1674 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:37:52.425273 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1211/2429] 10.222.0.49 () {58 vars in 1152 bytes} [Wed Jun 15 10:37:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:37:52.429289 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960e67fa-b9a2-4e5b-9844-b43c912aad1e None None] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:37:52.438436 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960e67fa-b9a2-4e5b-9844-b43c912aad1e None None] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:37:52.443139 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a1ca67-f6d0-490e-a914-61a6ac7ac303 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:37:52.443794 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a1ca67-f6d0-490e-a914-61a6ac7ac303 admin admin] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:37:52.444078 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a1ca67-f6d0-490e-a914-61a6ac7ac303 admin admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:37:52.444370 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a1ca67-f6d0-490e-a914-61a6ac7ac303 admin admin] PATH_INFO: `/v3/projects` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:37:52.445039 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a1ca67-f6d0-490e-a914-61a6ac7ac303 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:37:52.446939 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a1ca67-f6d0-490e-a914-61a6ac7ac303 admin admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:37:52.473234 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1219/2430] 10.222.0.49 () {60 vars in 1152 bytes} [Wed Jun 15 10:37:52 2022] POST /identity/v3/projects => generated 373 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 10:37:52.482682 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c46f72-3a93-4e2f-9a1f-d17c443e1674 admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:37:52.492960 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c46f72-3a93-4e2f-9a1f-d17c443e1674 admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:37:52.533458 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:37:52.552213 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ab2f4c-b3f8-424e-8171-8331e95b8c7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:37:52.552879 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ab2f4c-b3f8-424e-8171-8331e95b8c7e admin admin] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:37:52.553164 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ab2f4c-b3f8-424e-8171-8331e95b8c7e admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:37:52.553444 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ab2f4c-b3f8-424e-8171-8331e95b8c7e admin admin] PATH_INFO: `/v3/users` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:37:52.554152 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ab2f4c-b3f8-424e-8171-8331e95b8c7e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:37:52.556096 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ab2f4c-b3f8-424e-8171-8331e95b8c7e admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:37:52.575487 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1212/2431] 10.222.0.49 () {60 vars in 1143 bytes} [Wed Jun 15 10:37:52 2022] POST /identity/v3/users => generated 287 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 10:37:52.579576 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a1ca67-f6d0-490e-a914-61a6ac7ac303 admin admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:37:52.590815 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a1ca67-f6d0-490e-a914-61a6ac7ac303 admin admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:37:52.595506 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd639f26-e886-4076-aa04-cd237c917600 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:37:52.596774 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd639f26-e886-4076-aa04-cd237c917600 admin admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:37:52.597057 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd639f26-e886-4076-aa04-cd237c917600 admin admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:37:52.597334 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd639f26-e886-4076-aa04-cd237c917600 admin admin] PATH_INFO: `/v3/roles` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:37:52.598062 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd639f26-e886-4076-aa04-cd237c917600 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:37:52.599681 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd639f26-e886-4076-aa04-cd237c917600 admin admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:37:52.609979 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1220/2432] 10.222.0.49 () {58 vars in 1121 bytes} [Wed Jun 15 10:37:52 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 10:37:52.614531 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ab2f4c-b3f8-424e-8171-8331e95b8c7e admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:37:52.624584 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ab2f4c-b3f8-424e-8171-8331e95b8c7e admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:37:52.630946 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c618256-f1c2-4934-8178-1861ebbbe299 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:37:52.631642 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c618256-f1c2-4934-8178-1861ebbbe299 admin admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:37:52.631929 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c618256-f1c2-4934-8178-1861ebbbe299 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:37:52.632205 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c618256-f1c2-4934-8178-1861ebbbe299 admin admin] PATH_INFO: `/v3/roles` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:37:52.632930 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c618256-f1c2-4934-8178-1861ebbbe299 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:37:52.634530 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c618256-f1c2-4934-8178-1861ebbbe299 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:37:52.645081 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1213/2433] 10.222.0.49 () {58 vars in 1121 bytes} [Wed Jun 15 10:37:52 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 10:37:52.649111 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd639f26-e886-4076-aa04-cd237c917600 admin admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:37:52.660886 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd639f26-e886-4076-aa04-cd237c917600 admin admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:37:52.665575 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9ba67e-360b-4b32-b358-9213263f8811 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:37:52.666435 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9ba67e-360b-4b32-b358-9213263f8811 admin admin] REQUEST_METHOD: `PUT` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:37:52.666717 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9ba67e-360b-4b32-b358-9213263f8811 admin admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:37:52.666994 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9ba67e-360b-4b32-b358-9213263f8811 admin admin] PATH_INFO: `/v3/projects/6bbfa4912b4d4212a5ddbf0b419c530b/users/79e89f4ef7a34126978170ac9a633d43/roles/6876e57cc51c448097ab109d35b2a3da` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:37:52.682730 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9ba67e-360b-4b32-b358-9213263f8811 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6bbfa4912b4d4212a5ddbf0b419c530b, user_id=79e89f4ef7a34126978170ac9a633d43, role_id=6876e57cc51c448097ab109d35b2a3da)` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:37:52.685423 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9ba67e-360b-4b32-b358-9213263f8811 admin admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:37:52.698332 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1221/2434] 10.222.0.49 () {60 vars in 1482 bytes} [Wed Jun 15 10:37:52 2022] PUT /identity/v3/projects/6bbfa4912b4d4212a5ddbf0b419c530b/users/79e89f4ef7a34126978170ac9a633d43/roles/6876e57cc51c448097ab109d35b2a3da => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:37:52.708956 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bfd89b-3578-41ef-8951-23a2d936acc9 None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:37:52.709248 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bfd89b-3578-41ef-8951-23a2d936acc9 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:37:52.709526 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bfd89b-3578-41ef-8951-23a2d936acc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:37:52.735147 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-84bfd89b-3578-41ef-8951-23a2d936acc9 None None] MFA Rules not processed for user `79e89f4ef7a34126978170ac9a633d43`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:37:52.736383 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-84bfd89b-3578-41ef-8951-23a2d936acc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:37:52.801376 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1214/2435] 10.222.0.49 () {58 vars in 957 bytes} [Wed Jun 15 10:37:52 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:37:52.805874 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9ba67e-360b-4b32-b358-9213263f8811 admin admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:37:52.866414 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9ba67e-360b-4b32-b358-9213263f8811 admin admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:37:52.871428 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4a02ba-1c6c-498b-8cfc-39167901d067 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:37:52.872091 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4a02ba-1c6c-498b-8cfc-39167901d067 admin admin] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:37:52.872380 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4a02ba-1c6c-498b-8cfc-39167901d067 admin admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:37:52.872661 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4a02ba-1c6c-498b-8cfc-39167901d067 admin admin] PATH_INFO: `/v3/projects` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:37:52.873580 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb4a02ba-1c6c-498b-8cfc-39167901d067 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:37:52.875574 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb4a02ba-1c6c-498b-8cfc-39167901d067 admin admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:37:52.901514 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1222/2436] 10.222.0.49 () {60 vars in 1152 bytes} [Wed Jun 15 10:37:52 2022] POST /identity/v3/projects => generated 371 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 10:37:52.905512 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84bfd89b-3578-41ef-8951-23a2d936acc9 None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:37:52.916327 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84bfd89b-3578-41ef-8951-23a2d936acc9 None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:37:52.918203 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4a02ba-1c6c-498b-8cfc-39167901d067 admin admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:37:52.919963 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-cb4a02ba-1c6c-498b-8cfc-39167901d067 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:37:52.934059 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:37:52.975774 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65f3f2d-ad12-4d07-bb84-f4ff88a99d72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:37:52.976486 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65f3f2d-ad12-4d07-bb84-f4ff88a99d72 admin admin] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:37:52.976780 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65f3f2d-ad12-4d07-bb84-f4ff88a99d72 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:37:52.977059 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65f3f2d-ad12-4d07-bb84-f4ff88a99d72 admin admin] PATH_INFO: `/v3/users` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:37:52.977759 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65f3f2d-ad12-4d07-bb84-f4ff88a99d72 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:37:52.979738 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65f3f2d-ad12-4d07-bb84-f4ff88a99d72 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:37:53.001191 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1215/2437] 10.222.0.49 () {60 vars in 1143 bytes} [Wed Jun 15 10:37:52 2022] POST /identity/v3/users => generated 292 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 10:37:53.001536 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4a02ba-1c6c-498b-8cfc-39167901d067 admin admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:37:53.005389 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65f3f2d-ad12-4d07-bb84-f4ff88a99d72 admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:37:53.008008 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8156f9c8-5abe-4c45-93cf-32be11b91365 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ac54528370d4747b6e8777fba9d0c82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '559b3e713f5844b6ae37151d56dd419f', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:37:53.008659 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8156f9c8-5abe-4c45-93cf-32be11b91365 service nova] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:37:53.008947 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8156f9c8-5abe-4c45-93cf-32be11b91365 service nova] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:37:53.009220 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8156f9c8-5abe-4c45-93cf-32be11b91365 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:37:53.011263 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-8156f9c8-5abe-4c45-93cf-32be11b91365 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:37:53.015258 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65f3f2d-ad12-4d07-bb84-f4ff88a99d72 admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:37:53.020756 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818bea58-4ba3-455f-a4ca-0f1728c94be8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:37:53.021523 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818bea58-4ba3-455f-a4ca-0f1728c94be8 admin admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:37:53.021815 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818bea58-4ba3-455f-a4ca-0f1728c94be8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:37:53.022103 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818bea58-4ba3-455f-a4ca-0f1728c94be8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:37:53.022832 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-818bea58-4ba3-455f-a4ca-0f1728c94be8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:37:53.024894 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-818bea58-4ba3-455f-a4ca-0f1728c94be8 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:37:53.035501 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1216/2438] 10.222.0.49 () {58 vars in 1121 bytes} [Wed Jun 15 10:37:53 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 15 10:37:53.039334 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818bea58-4ba3-455f-a4ca-0f1728c94be8 admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:37:53.048810 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818bea58-4ba3-455f-a4ca-0f1728c94be8 admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:37:53.054023 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a79e27-98da-4da6-8569-6dd9b367d427 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:37:53.054917 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a79e27-98da-4da6-8569-6dd9b367d427 admin admin] REQUEST_METHOD: `PUT` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:37:53.055238 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a79e27-98da-4da6-8569-6dd9b367d427 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:37:53.055519 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a79e27-98da-4da6-8569-6dd9b367d427 admin admin] PATH_INFO: `/v3/projects/b2dcb7cbcb4948a9aac17180fe554641/users/f9ab03b311a54587b09a7f8732be9d1f/roles/8051174e7a7947d5b991ebd793c2e543` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:37:53.070995 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a79e27-98da-4da6-8569-6dd9b367d427 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b2dcb7cbcb4948a9aac17180fe554641, user_id=f9ab03b311a54587b09a7f8732be9d1f, role_id=8051174e7a7947d5b991ebd793c2e543)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:37:53.072037 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8156f9c8-5abe-4c45-93cf-32be11b91365 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:37:53.073663 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a79e27-98da-4da6-8569-6dd9b367d427 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:37:53.073800 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8156f9c8-5abe-4c45-93cf-32be11b91365 service nova] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:37:53.083914 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1223/2439] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Jun 15 10:37:52 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 10:37:53.086594 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1217/2440] 10.222.0.49 () {60 vars in 1482 bytes} [Wed Jun 15 10:37:53 2022] PUT /identity/v3/projects/b2dcb7cbcb4948a9aac17180fe554641/users/f9ab03b311a54587b09a7f8732be9d1f/roles/8051174e7a7947d5b991ebd793c2e543 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 10:37:53.090673 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8156f9c8-5abe-4c45-93cf-32be11b91365 service nova] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:37:53.100977 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8156f9c8-5abe-4c45-93cf-32be11b91365 service nova] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:37:53.158103 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d7af46-d794-4ca1-b3d2-0b2c617c2f7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:37:53.159109 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d7af46-d794-4ca1-b3d2-0b2c617c2f7a admin admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:37:53.159398 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d7af46-d794-4ca1-b3d2-0b2c617c2f7a admin admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:37:53.159675 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d7af46-d794-4ca1-b3d2-0b2c617c2f7a admin admin] PATH_INFO: `/v3/roles` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:37:53.160405 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d7af46-d794-4ca1-b3d2-0b2c617c2f7a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:37:53.161996 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d7af46-d794-4ca1-b3d2-0b2c617c2f7a admin admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:37:53.175425 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1224/2441] 10.222.0.49 () {58 vars in 1121 bytes} [Wed Jun 15 10:37:53 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 10:37:53.176750 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a79e27-98da-4da6-8569-6dd9b367d427 admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:37:53.183563 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:37:53.188157 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a79e27-98da-4da6-8569-6dd9b367d427 admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:37:53.195642 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a717ec6-b5c1-4ea7-9bf1-52b643348e7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:37:53.196531 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a717ec6-b5c1-4ea7-9bf1-52b643348e7f admin admin] REQUEST_METHOD: `PUT` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:37:53.196867 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a717ec6-b5c1-4ea7-9bf1-52b643348e7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:37:53.197085 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a717ec6-b5c1-4ea7-9bf1-52b643348e7f admin admin] PATH_INFO: `/v3/projects/b2dcb7cbcb4948a9aac17180fe554641/users/f9ab03b311a54587b09a7f8732be9d1f/roles/8051174e7a7947d5b991ebd793c2e543` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:37:53.199074 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a717ec6-b5c1-4ea7-9bf1-52b643348e7f admin admin] RBAC: Authorizing `identity:create_grant(project_id=b2dcb7cbcb4948a9aac17180fe554641, user_id=f9ab03b311a54587b09a7f8732be9d1f, role_id=8051174e7a7947d5b991ebd793c2e543)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:37:53.201746 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a717ec6-b5c1-4ea7-9bf1-52b643348e7f admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:37:53.212547 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1218/2442] 10.222.0.49 () {60 vars in 1482 bytes} [Wed Jun 15 10:37:53 2022] PUT /identity/v3/projects/b2dcb7cbcb4948a9aac17180fe554641/users/f9ab03b311a54587b09a7f8732be9d1f/roles/8051174e7a7947d5b991ebd793c2e543 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:37:53.224395 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0e015b-9aeb-4280-bd59-36ba84fc1153 None None] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:37:53.224693 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0e015b-9aeb-4280-bd59-36ba84fc1153 None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:37:53.224976 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0e015b-9aeb-4280-bd59-36ba84fc1153 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:37:53.254287 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.auth.core [None req-1f0e015b-9aeb-4280-bd59-36ba84fc1153 None None] MFA Rules not processed for user `f9ab03b311a54587b09a7f8732be9d1f`. Rule list: `[]` (Enabled: `True`). {{(pid=80119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:37:53.255509 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-1f0e015b-9aeb-4280-bd59-36ba84fc1153 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:37:53.306761 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=100 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:37:53.358957 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1225/2443] 10.222.0.49 () {58 vars in 957 bytes} [Wed Jun 15 10:37:53 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:37:53.393456 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a717ec6-b5c1-4ea7-9bf1-52b643348e7f admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:37:53.395691 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-8a717ec6-b5c1-4ea7-9bf1-52b643348e7f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:37:53.462360 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a717ec6-b5c1-4ea7-9bf1-52b643348e7f admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:37:53.466564 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9cc15b0-7ae4-453b-8416-8ee07dd3b88e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20d1327553404b80aecee87c78fcb52b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '559b3e713f5844b6ae37151d56dd419f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:37:53.467293 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9cc15b0-7ae4-453b-8416-8ee07dd3b88e service cinder] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:37:53.467587 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9cc15b0-7ae4-453b-8416-8ee07dd3b88e service cinder] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:37:53.467869 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9cc15b0-7ae4-453b-8416-8ee07dd3b88e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:37:53.483127 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9cc15b0-7ae4-453b-8416-8ee07dd3b88e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:37:53.484564 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9cc15b0-7ae4-453b-8416-8ee07dd3b88e service cinder] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:37:53.495675 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1219/2444] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Jun 15 10:37:53 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:37:53.877466 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0e015b-9aeb-4280-bd59-36ba84fc1153 None None] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:37:53.940361 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0e015b-9aeb-4280-bd59-36ba84fc1153 None None] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:37:53.945094 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6b30e7-1243-4e1e-95a9-262754fcfdbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:37:53.946016 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6b30e7-1243-4e1e-95a9-262754fcfdbb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:37:53.946326 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6b30e7-1243-4e1e-95a9-262754fcfdbb admin admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:37:53.946727 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6b30e7-1243-4e1e-95a9-262754fcfdbb admin admin] PATH_INFO: `/v3/users/79e89f4ef7a34126978170ac9a633d43` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:37:53.947970 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6b30e7-1243-4e1e-95a9-262754fcfdbb admin admin] RBAC: Authorizing `identity:delete_user(user_id=79e89f4ef7a34126978170ac9a633d43)` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:37:53.950084 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6b30e7-1243-4e1e-95a9-262754fcfdbb admin admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:37:53.998590 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-9a6b30e7-1243-4e1e-95a9-262754fcfdbb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '79e89f4ef7a34126978170ac9a633d43'} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:37:54.005684 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-9a6b30e7-1243-4e1e-95a9-262754fcfdbb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '79e89f4ef7a34126978170ac9a633d43'} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:37:54.006362 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-9a6b30e7-1243-4e1e-95a9-262754fcfdbb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '79e89f4ef7a34126978170ac9a633d43'} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:37:54.006953 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-9a6b30e7-1243-4e1e-95a9-262754fcfdbb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '79e89f4ef7a34126978170ac9a633d43'} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:37:54.013886 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-9a6b30e7-1243-4e1e-95a9-262754fcfdbb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '79e89f4ef7a34126978170ac9a633d43'} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:37:54.034035 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1226/2445] 10.222.0.49 () {58 vars in 1223 bytes} [Wed Jun 15 10:37:53 2022] DELETE /identity/v3/users/79e89f4ef7a34126978170ac9a633d43 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:37:54.038514 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5087515-979c-4fe1-87e8-4e9d8243c4ec None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:37:54.039004 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5087515-979c-4fe1-87e8-4e9d8243c4ec None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:37:54.039117 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5087515-979c-4fe1-87e8-4e9d8243c4ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:37:54.056493 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-d5087515-979c-4fe1-87e8-4e9d8243c4ec None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:37:54.057747 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-d5087515-979c-4fe1-87e8-4e9d8243c4ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:37:54.118362 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1220/2446] 10.222.0.49 () {58 vars in 957 bytes} [Wed Jun 15 10:37:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:37:54.128641 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6b30e7-1243-4e1e-95a9-262754fcfdbb admin admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:37:54.130072 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-9a6b30e7-1243-4e1e-95a9-262754fcfdbb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:37:54.185364 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6b30e7-1243-4e1e-95a9-262754fcfdbb admin admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:37:54.189003 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c661a6e-8870-4527-bdd4-9f9fdd8376c3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0cb44a17d205431793e9a8791e477a32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '559b3e713f5844b6ae37151d56dd419f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:37:54.190269 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c661a6e-8870-4527-bdd4-9f9fdd8376c3 service neutron] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:37:54.190607 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c661a6e-8870-4527-bdd4-9f9fdd8376c3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:37:54.190831 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c661a6e-8870-4527-bdd4-9f9fdd8376c3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:37:54.196737 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:37:54.201331 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c661a6e-8870-4527-bdd4-9f9fdd8376c3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:37:54.204231 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c661a6e-8870-4527-bdd4-9f9fdd8376c3 service neutron] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:37:54.214352 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1227/2447] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Jun 15 10:37:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:37:54.478771 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5087515-979c-4fe1-87e8-4e9d8243c4ec None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:37:54.480310 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-d5087515-979c-4fe1-87e8-4e9d8243c4ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:37:54.495800 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5087515-979c-4fe1-87e8-4e9d8243c4ec None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:37:54.501482 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46fbc808-4073-406c-bda5-4b0c2b52a7cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:37:54.502436 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fbc808-4073-406c-bda5-4b0c2b52a7cc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:37:54.502861 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fbc808-4073-406c-bda5-4b0c2b52a7cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:37:54.502961 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fbc808-4073-406c-bda5-4b0c2b52a7cc admin admin] PATH_INFO: `/v3/projects/6bbfa4912b4d4212a5ddbf0b419c530b` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:37:54.504528 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46fbc808-4073-406c-bda5-4b0c2b52a7cc admin admin] RBAC: Authorizing `identity:delete_project(project_id=6bbfa4912b4d4212a5ddbf0b419c530b)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:37:54.506401 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46fbc808-4073-406c-bda5-4b0c2b52a7cc admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:37:54.554470 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-46fbc808-4073-406c-bda5-4b0c2b52a7cc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6bbfa4912b4d4212a5ddbf0b419c530b', 'request_id': 'req-46fbc808-4073-406c-bda5-4b0c2b52a7cc'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:37:54.562624 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-46fbc808-4073-406c-bda5-4b0c2b52a7cc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6bbfa4912b4d4212a5ddbf0b419c530b', 'request_id': 'req-46fbc808-4073-406c-bda5-4b0c2b52a7cc'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:37:54.569503 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-46fbc808-4073-406c-bda5-4b0c2b52a7cc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6bbfa4912b4d4212a5ddbf0b419c530b', 'request_id': 'req-46fbc808-4073-406c-bda5-4b0c2b52a7cc'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:37:54.581495 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-46fbc808-4073-406c-bda5-4b0c2b52a7cc admin admin] The token cache is being invalidate because project 6bbfa4912b4d4212a5ddbf0b419c530b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80120) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 10:37:54.581778 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-46fbc808-4073-406c-bda5-4b0c2b52a7cc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:37:54.582405 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-46fbc808-4073-406c-bda5-4b0c2b52a7cc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:37:54.584796 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1221/2448] 10.222.0.49 () {58 vars in 1232 bytes} [Wed Jun 15 10:37:54 2022] DELETE /identity/v3/projects/6bbfa4912b4d4212a5ddbf0b419c530b => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:37:54.589485 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c661a6e-8870-4527-bdd4-9f9fdd8376c3 service neutron] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:37:54.599835 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c661a6e-8870-4527-bdd4-9f9fdd8376c3 service neutron] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:37:54.659066 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e390d6af-a4a2-427e-839d-0c5a5ce7c40b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:37:54.659997 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e390d6af-a4a2-427e-839d-0c5a5ce7c40b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:37:54.660274 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e390d6af-a4a2-427e-839d-0c5a5ce7c40b admin admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:37:54.660545 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e390d6af-a4a2-427e-839d-0c5a5ce7c40b admin admin] PATH_INFO: `/v3/users/f9ab03b311a54587b09a7f8732be9d1f` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:37:54.661719 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e390d6af-a4a2-427e-839d-0c5a5ce7c40b admin admin] RBAC: Authorizing `identity:delete_user(user_id=f9ab03b311a54587b09a7f8732be9d1f)` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:37:54.663669 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e390d6af-a4a2-427e-839d-0c5a5ce7c40b admin admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:37:54.716070 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-e390d6af-a4a2-427e-839d-0c5a5ce7c40b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f9ab03b311a54587b09a7f8732be9d1f'} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:37:54.725106 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-e390d6af-a4a2-427e-839d-0c5a5ce7c40b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f9ab03b311a54587b09a7f8732be9d1f'} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:37:54.726027 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-e390d6af-a4a2-427e-839d-0c5a5ce7c40b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f9ab03b311a54587b09a7f8732be9d1f'} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:37:54.726938 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-e390d6af-a4a2-427e-839d-0c5a5ce7c40b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f9ab03b311a54587b09a7f8732be9d1f'} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:37:54.736519 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-e390d6af-a4a2-427e-839d-0c5a5ce7c40b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f9ab03b311a54587b09a7f8732be9d1f'} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:37:54.764633 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1228/2449] 10.222.0.49 () {58 vars in 1223 bytes} [Wed Jun 15 10:37:54 2022] DELETE /identity/v3/users/f9ab03b311a54587b09a7f8732be9d1f => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:37:54.788197 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46fbc808-4073-406c-bda5-4b0c2b52a7cc admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:37:54.790114 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-46fbc808-4073-406c-bda5-4b0c2b52a7cc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:37:54.809959 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46fbc808-4073-406c-bda5-4b0c2b52a7cc admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:37:54.823749 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:37:54.891668 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b786fc02-e5d5-4242-b211-238ec74052ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:37:54.893380 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b786fc02-e5d5-4242-b211-238ec74052ed admin admin] REQUEST_METHOD: `DELETE` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:37:54.893832 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b786fc02-e5d5-4242-b211-238ec74052ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:37:54.894248 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b786fc02-e5d5-4242-b211-238ec74052ed admin admin] PATH_INFO: `/v3/users/b0aad8a3cc054e2f98ed82488fe0a034` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:37:54.895521 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b786fc02-e5d5-4242-b211-238ec74052ed admin admin] RBAC: Authorizing `identity:delete_user(user_id=b0aad8a3cc054e2f98ed82488fe0a034)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:37:54.897455 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b786fc02-e5d5-4242-b211-238ec74052ed admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:37:54.952944 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-b786fc02-e5d5-4242-b211-238ec74052ed admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b0aad8a3cc054e2f98ed82488fe0a034'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:37:54.961912 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-b786fc02-e5d5-4242-b211-238ec74052ed admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b0aad8a3cc054e2f98ed82488fe0a034'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:37:54.962840 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-b786fc02-e5d5-4242-b211-238ec74052ed admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b0aad8a3cc054e2f98ed82488fe0a034'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:37:54.963682 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-b786fc02-e5d5-4242-b211-238ec74052ed admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b0aad8a3cc054e2f98ed82488fe0a034'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:37:54.974225 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-b786fc02-e5d5-4242-b211-238ec74052ed admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b0aad8a3cc054e2f98ed82488fe0a034'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:37:54.995880 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1222/2450] 10.222.0.49 () {58 vars in 1223 bytes} [Wed Jun 15 10:37:54 2022] DELETE /identity/v3/users/b0aad8a3cc054e2f98ed82488fe0a034 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:37:55.000447 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fb89e3-cef6-4ce2-9102-3504eda845dc None None] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:37:55.000821 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fb89e3-cef6-4ce2-9102-3504eda845dc None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:37:55.001175 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fb89e3-cef6-4ce2-9102-3504eda845dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:37:55.020297 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.auth.core [None req-50fb89e3-cef6-4ce2-9102-3504eda845dc None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:37:55.021787 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-50fb89e3-cef6-4ce2-9102-3504eda845dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:37:55.075837 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b786fc02-e5d5-4242-b211-238ec74052ed admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:37:55.077379 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-b786fc02-e5d5-4242-b211-238ec74052ed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:37:55.084121 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1229/2451] 10.222.0.49 () {58 vars in 957 bytes} [Wed Jun 15 10:37:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:37:55.095663 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b786fc02-e5d5-4242-b211-238ec74052ed admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:37:55.095840 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50fb89e3-cef6-4ce2-9102-3504eda845dc None None] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:37:55.096992 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-50fb89e3-cef6-4ce2-9102-3504eda845dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:37:55.100878 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6590a2f-aaa2-44d5-9f87-4cb8c58094db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:37:55.108490 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6590a2f-aaa2-44d5-9f87-4cb8c58094db admin admin] REQUEST_METHOD: `DELETE` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:37:55.108778 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6590a2f-aaa2-44d5-9f87-4cb8c58094db admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:37:55.109055 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6590a2f-aaa2-44d5-9f87-4cb8c58094db admin admin] PATH_INFO: `/v3/projects/b2dcb7cbcb4948a9aac17180fe554641` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:37:55.110327 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6590a2f-aaa2-44d5-9f87-4cb8c58094db admin admin] RBAC: Authorizing `identity:delete_project(project_id=b2dcb7cbcb4948a9aac17180fe554641)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:37:55.111797 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:37:55.112316 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6590a2f-aaa2-44d5-9f87-4cb8c58094db admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:37:55.158140 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50fb89e3-cef6-4ce2-9102-3504eda845dc None None] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:37:55.161553 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-b6590a2f-aaa2-44d5-9f87-4cb8c58094db admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b2dcb7cbcb4948a9aac17180fe554641', 'request_id': 'req-b6590a2f-aaa2-44d5-9f87-4cb8c58094db'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:37:55.168912 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-b6590a2f-aaa2-44d5-9f87-4cb8c58094db admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b2dcb7cbcb4948a9aac17180fe554641', 'request_id': 'req-b6590a2f-aaa2-44d5-9f87-4cb8c58094db'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:37:55.175538 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-b6590a2f-aaa2-44d5-9f87-4cb8c58094db admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b2dcb7cbcb4948a9aac17180fe554641', 'request_id': 'req-b6590a2f-aaa2-44d5-9f87-4cb8c58094db'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:37:55.186923 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-b6590a2f-aaa2-44d5-9f87-4cb8c58094db admin admin] The token cache is being invalidate because project b2dcb7cbcb4948a9aac17180fe554641 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80120) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 10:37:55.187251 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-b6590a2f-aaa2-44d5-9f87-4cb8c58094db admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:37:55.187922 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-b6590a2f-aaa2-44d5-9f87-4cb8c58094db admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:37:55.190298 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1223/2452] 10.222.0.49 () {58 vars in 1232 bytes} [Wed Jun 15 10:37:55 2022] DELETE /identity/v3/projects/b2dcb7cbcb4948a9aac17180fe554641 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 10:37:55.194716 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ccb18e-48e4-4ac1-aa1f-081b3b28b89e None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:37:55.195008 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ccb18e-48e4-4ac1-aa1f-081b3b28b89e None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:37:55.195310 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ccb18e-48e4-4ac1-aa1f-081b3b28b89e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:37:55.210668 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911f79f4-54fd-478f-a167-9dd46ec10b96 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0cb44a17d205431793e9a8791e477a32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '559b3e713f5844b6ae37151d56dd419f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:37:55.211514 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911f79f4-54fd-478f-a167-9dd46ec10b96 service neutron] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:37:55.211806 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911f79f4-54fd-478f-a167-9dd46ec10b96 service neutron] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:37:55.212081 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911f79f4-54fd-478f-a167-9dd46ec10b96 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:37:55.213904 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-911f79f4-54fd-478f-a167-9dd46ec10b96 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:37:55.214683 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-77ccb18e-48e4-4ac1-aa1f-081b3b28b89e None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:37:55.215897 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-77ccb18e-48e4-4ac1-aa1f-081b3b28b89e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:37:55.228730 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:37:55.277375 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-911f79f4-54fd-478f-a167-9dd46ec10b96 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:37:55.277683 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1224/2453] 10.222.0.49 () {58 vars in 957 bytes} [Wed Jun 15 10:37:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 10:37:55.280239 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-911f79f4-54fd-478f-a167-9dd46ec10b96 service neutron] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:37:55.285331 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51c730b-dc95-48ea-b318-3f3a0c099394 None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:37:55.285692 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51c730b-dc95-48ea-b318-3f3a0c099394 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:37:55.285978 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51c730b-dc95-48ea-b318-3f3a0c099394 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:37:55.290590 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1230/2454] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Jun 15 10:37:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 196 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 10:37:55.305059 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-d51c730b-dc95-48ea-b318-3f3a0c099394 None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:37:55.306214 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-d51c730b-dc95-48ea-b318-3f3a0c099394 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:37:55.314231 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1225/2455] 10.222.0.49 () {58 vars in 957 bytes} [Wed Jun 15 10:37:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 10:37:55.319119 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911f79f4-54fd-478f-a167-9dd46ec10b96 service neutron] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:37:55.329505 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911f79f4-54fd-478f-a167-9dd46ec10b96 service neutron] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:37:55.334505 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86bf380c-cff6-46eb-99dd-4ea43ecc981a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:37:55.335259 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bf380c-cff6-46eb-99dd-4ea43ecc981a admin admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:37:55.335548 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bf380c-cff6-46eb-99dd-4ea43ecc981a admin admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:37:55.335852 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bf380c-cff6-46eb-99dd-4ea43ecc981a admin admin] PATH_INFO: `/v3/domains` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:37:55.336624 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86bf380c-cff6-46eb-99dd-4ea43ecc981a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:37:55.336932 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86bf380c-cff6-46eb-99dd-4ea43ecc981a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:37:55.338683 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86bf380c-cff6-46eb-99dd-4ea43ecc981a admin admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:37:55.352428 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1231/2456] 10.222.0.49 () {58 vars in 1152 bytes} [Wed Jun 15 10:37:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:37:55.357383 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51c730b-dc95-48ea-b318-3f3a0c099394 None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:37:55.367642 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51c730b-dc95-48ea-b318-3f3a0c099394 None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:37:55.372961 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5f6b51-3c76-4a7a-9cb1-07658e60f5a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:37:55.373625 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5f6b51-3c76-4a7a-9cb1-07658e60f5a5 admin admin] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:37:55.373948 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5f6b51-3c76-4a7a-9cb1-07658e60f5a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:37:55.374222 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5f6b51-3c76-4a7a-9cb1-07658e60f5a5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:37:55.374901 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd5f6b51-3c76-4a7a-9cb1-07658e60f5a5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:37:55.376872 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd5f6b51-3c76-4a7a-9cb1-07658e60f5a5 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:37:55.428064 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1226/2457] 10.222.0.49 () {60 vars in 1152 bytes} [Wed Jun 15 10:37:55 2022] POST /identity/v3/projects => generated 355 bytes in 72 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 10:37:55.428623 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86bf380c-cff6-46eb-99dd-4ea43ecc981a admin admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:37:55.439317 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86bf380c-cff6-46eb-99dd-4ea43ecc981a admin admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:37:55.499023 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10cba2de-d40c-4510-b029-ebccb101a4e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:37:55.499729 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10cba2de-d40c-4510-b029-ebccb101a4e8 admin admin] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:37:55.500017 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10cba2de-d40c-4510-b029-ebccb101a4e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:37:55.500293 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10cba2de-d40c-4510-b029-ebccb101a4e8 admin admin] PATH_INFO: `/v3/users` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:37:55.501006 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10cba2de-d40c-4510-b029-ebccb101a4e8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:37:55.503642 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10cba2de-d40c-4510-b029-ebccb101a4e8 admin admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:37:55.514263 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:37:55.528214 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1232/2458] 10.222.0.49 () {60 vars in 1143 bytes} [Wed Jun 15 10:37:55 2022] POST /identity/v3/users => generated 278 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 10:37:55.532039 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5f6b51-3c76-4a7a-9cb1-07658e60f5a5 admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:37:55.543037 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5f6b51-3c76-4a7a-9cb1-07658e60f5a5 admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:37:55.548184 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b70b17-a167-4223-9f71-24c44a0297d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:37:55.549468 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b70b17-a167-4223-9f71-24c44a0297d9 admin admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:37:55.549836 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b70b17-a167-4223-9f71-24c44a0297d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:37:55.550182 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b70b17-a167-4223-9f71-24c44a0297d9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:37:55.550987 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b70b17-a167-4223-9f71-24c44a0297d9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:37:55.552718 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b70b17-a167-4223-9f71-24c44a0297d9 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:37:55.567119 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1227/2459] 10.222.0.49 () {58 vars in 1121 bytes} [Wed Jun 15 10:37:55 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 15 10:37:55.570348 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10cba2de-d40c-4510-b029-ebccb101a4e8 admin admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:37:55.579179 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b70b17-a167-4223-9f71-24c44a0297d9 admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:37:55.580930 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-c3b70b17-a167-4223-9f71-24c44a0297d9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:37:55.581189 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10cba2de-d40c-4510-b029-ebccb101a4e8 admin admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:37:55.586647 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91cf5143-6ceb-49d0-b77a-24b892a512b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:37:55.587362 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91cf5143-6ceb-49d0-b77a-24b892a512b7 admin admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:37:55.587677 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91cf5143-6ceb-49d0-b77a-24b892a512b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:37:55.588162 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91cf5143-6ceb-49d0-b77a-24b892a512b7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:37:55.588907 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91cf5143-6ceb-49d0-b77a-24b892a512b7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:37:55.590658 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91cf5143-6ceb-49d0-b77a-24b892a512b7 admin admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:37:55.602909 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1233/2460] 10.222.0.49 () {58 vars in 1121 bytes} [Wed Jun 15 10:37:55 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 10:37:55.604184 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b70b17-a167-4223-9f71-24c44a0297d9 admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:37:55.607109 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91cf5143-6ceb-49d0-b77a-24b892a512b7 admin admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:37:55.610906 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18e8303-1f9b-4433-bd49-70a62859d40d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:37:55.612437 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18e8303-1f9b-4433-bd49-70a62859d40d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:37:55.612845 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18e8303-1f9b-4433-bd49-70a62859d40d admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:37:55.613218 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18e8303-1f9b-4433-bd49-70a62859d40d admin admin] PATH_INFO: `/v3/projects/369db092ec72463b9ff27caed4e828f8` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:37:55.614519 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a18e8303-1f9b-4433-bd49-70a62859d40d admin admin] RBAC: Authorizing `identity:delete_project(project_id=369db092ec72463b9ff27caed4e828f8)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:37:55.616805 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a18e8303-1f9b-4433-bd49-70a62859d40d admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:37:55.618437 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91cf5143-6ceb-49d0-b77a-24b892a512b7 admin admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:37:55.623238 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac52e74c-7c3f-4534-a9c8-31f63a6a8b6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:37:55.624271 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac52e74c-7c3f-4534-a9c8-31f63a6a8b6a admin admin] REQUEST_METHOD: `PUT` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:37:55.624433 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac52e74c-7c3f-4534-a9c8-31f63a6a8b6a admin admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:37:55.624719 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac52e74c-7c3f-4534-a9c8-31f63a6a8b6a admin admin] PATH_INFO: `/v3/projects/47ef8e14002f446f98ae1c5ac1b11370/users/205068af1e284d7fb7f0d4a93b0cdb3c/roles/6876e57cc51c448097ab109d35b2a3da` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:37:55.641616 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac52e74c-7c3f-4534-a9c8-31f63a6a8b6a admin admin] RBAC: Authorizing `identity:create_grant(project_id=47ef8e14002f446f98ae1c5ac1b11370, user_id=205068af1e284d7fb7f0d4a93b0cdb3c, role_id=6876e57cc51c448097ab109d35b2a3da)` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:37:55.644482 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac52e74c-7c3f-4534-a9c8-31f63a6a8b6a admin admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:37:55.657676 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1234/2461] 10.222.0.49 () {60 vars in 1482 bytes} [Wed Jun 15 10:37:55 2022] PUT /identity/v3/projects/47ef8e14002f446f98ae1c5ac1b11370/users/205068af1e284d7fb7f0d4a93b0cdb3c/roles/6876e57cc51c448097ab109d35b2a3da => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 10:37:55.667825 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc35c74b-326e-4621-b8b5-ca34165bbd53 None None] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:37:55.668224 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc35c74b-326e-4621-b8b5-ca34165bbd53 None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:37:55.668574 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc35c74b-326e-4621-b8b5-ca34165bbd53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:37:55.671925 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-a18e8303-1f9b-4433-bd49-70a62859d40d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '369db092ec72463b9ff27caed4e828f8', 'request_id': 'req-a18e8303-1f9b-4433-bd49-70a62859d40d'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:37:55.684593 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-a18e8303-1f9b-4433-bd49-70a62859d40d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '369db092ec72463b9ff27caed4e828f8', 'request_id': 'req-a18e8303-1f9b-4433-bd49-70a62859d40d'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:37:55.691902 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-a18e8303-1f9b-4433-bd49-70a62859d40d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '369db092ec72463b9ff27caed4e828f8', 'request_id': 'req-a18e8303-1f9b-4433-bd49-70a62859d40d'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:37:55.702847 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.auth.core [None req-bc35c74b-326e-4621-b8b5-ca34165bbd53 None None] MFA Rules not processed for user `205068af1e284d7fb7f0d4a93b0cdb3c`. Rule list: `[]` (Enabled: `True`). {{(pid=80119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:37:55.704773 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-bc35c74b-326e-4621-b8b5-ca34165bbd53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:37:55.705095 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-a18e8303-1f9b-4433-bd49-70a62859d40d admin admin] The token cache is being invalidate because project 369db092ec72463b9ff27caed4e828f8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80120) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 10:37:55.705548 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-a18e8303-1f9b-4433-bd49-70a62859d40d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:37:55.706527 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-a18e8303-1f9b-4433-bd49-70a62859d40d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:37:55.709345 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1228/2462] 10.222.0.49 () {58 vars in 1232 bytes} [Wed Jun 15 10:37:55 2022] DELETE /identity/v3/projects/369db092ec72463b9ff27caed4e828f8 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 10:37:55.713398 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18e8303-1f9b-4433-bd49-70a62859d40d admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:37:55.715308 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-a18e8303-1f9b-4433-bd49-70a62859d40d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:37:55.774862 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1235/2463] 10.222.0.49 () {58 vars in 957 bytes} [Wed Jun 15 10:37:55 2022] POST /identity/v3/auth/tokens => generated 3222 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 10:37:55.779528 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc35c74b-326e-4621-b8b5-ca34165bbd53 None None] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:37:55.780915 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-bc35c74b-326e-4621-b8b5-ca34165bbd53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:37:55.781274 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18e8303-1f9b-4433-bd49-70a62859d40d admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:37:55.786630 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46b0680-4ed5-4437-935a-aced24adc39c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:37:55.787535 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46b0680-4ed5-4437-935a-aced24adc39c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:37:55.787824 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46b0680-4ed5-4437-935a-aced24adc39c admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:37:55.788116 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46b0680-4ed5-4437-935a-aced24adc39c admin admin] PATH_INFO: `/v3/users/349b4db3fc1a40b487a87f58034f7013` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:37:55.793397 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e46b0680-4ed5-4437-935a-aced24adc39c admin admin] RBAC: Authorizing `identity:delete_user(user_id=349b4db3fc1a40b487a87f58034f7013)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:37:55.793397 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e46b0680-4ed5-4437-935a-aced24adc39c admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:37:55.795944 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc35c74b-326e-4621-b8b5-ca34165bbd53 None None] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:37:55.798021 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:37:55.801491 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd494d2b-3991-490d-83f3-f7c23be6a321 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:37:55.802161 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd494d2b-3991-490d-83f3-f7c23be6a321 admin admin] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:37:55.802450 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd494d2b-3991-490d-83f3-f7c23be6a321 admin admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:37:55.802734 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd494d2b-3991-490d-83f3-f7c23be6a321 admin admin] PATH_INFO: `/v3/projects` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:37:55.803476 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd494d2b-3991-490d-83f3-f7c23be6a321 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:37:55.805528 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd494d2b-3991-490d-83f3-f7c23be6a321 admin admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:37:55.834496 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1236/2464] 10.222.0.49 () {60 vars in 1152 bytes} [Wed Jun 15 10:37:55 2022] POST /identity/v3/projects => generated 353 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 10:37:55.835534 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd494d2b-3991-490d-83f3-f7c23be6a321 admin admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:37:55.848151 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-e46b0680-4ed5-4437-935a-aced24adc39c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '349b4db3fc1a40b487a87f58034f7013'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:37:55.855478 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-e46b0680-4ed5-4437-935a-aced24adc39c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '349b4db3fc1a40b487a87f58034f7013'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:37:55.856441 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-e46b0680-4ed5-4437-935a-aced24adc39c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '349b4db3fc1a40b487a87f58034f7013'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:37:55.857072 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-e46b0680-4ed5-4437-935a-aced24adc39c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '349b4db3fc1a40b487a87f58034f7013'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:37:55.865837 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-e46b0680-4ed5-4437-935a-aced24adc39c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '349b4db3fc1a40b487a87f58034f7013'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:37:55.886601 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1229/2465] 10.222.0.49 () {58 vars in 1223 bytes} [Wed Jun 15 10:37:55 2022] DELETE /identity/v3/users/349b4db3fc1a40b487a87f58034f7013 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:37:55.904275 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd494d2b-3991-490d-83f3-f7c23be6a321 admin admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:37:55.937967 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:37:55.965922 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a7349a-9104-444c-9322-4d644cf03d9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:37:55.966598 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a7349a-9104-444c-9322-4d644cf03d9b admin admin] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:37:55.966894 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a7349a-9104-444c-9322-4d644cf03d9b admin admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:37:55.967645 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a7349a-9104-444c-9322-4d644cf03d9b admin admin] PATH_INFO: `/v3/users` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:37:55.967925 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a7349a-9104-444c-9322-4d644cf03d9b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:37:55.969943 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a7349a-9104-444c-9322-4d644cf03d9b admin admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:37:55.995026 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1237/2466] 10.222.0.49 () {60 vars in 1143 bytes} [Wed Jun 15 10:37:55 2022] POST /identity/v3/users => generated 283 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 10:37:55.996206 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46b0680-4ed5-4437-935a-aced24adc39c admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:37:55.997692 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-e46b0680-4ed5-4437-935a-aced24adc39c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:37:56.010658 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46b0680-4ed5-4437-935a-aced24adc39c admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:37:56.015271 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dce8f1b-0e3d-4abc-ae73-e848546ad1ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:37:56.015934 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dce8f1b-0e3d-4abc-ae73-e848546ad1ab admin admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:37:56.016218 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dce8f1b-0e3d-4abc-ae73-e848546ad1ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:37:56.016490 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dce8f1b-0e3d-4abc-ae73-e848546ad1ab admin admin] PATH_INFO: `/v3/roles` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:37:56.017221 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dce8f1b-0e3d-4abc-ae73-e848546ad1ab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:37:56.018849 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dce8f1b-0e3d-4abc-ae73-e848546ad1ab admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:37:56.029601 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1230/2467] 10.222.0.49 () {58 vars in 1121 bytes} [Wed Jun 15 10:37:55 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 10:37:56.034063 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a7349a-9104-444c-9322-4d644cf03d9b admin admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:37:56.046222 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a7349a-9104-444c-9322-4d644cf03d9b admin admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:37:56.051830 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5bb589a-2aa0-4d5e-9a24-b5072f572353 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:37:56.052717 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bb589a-2aa0-4d5e-9a24-b5072f572353 admin admin] REQUEST_METHOD: `PUT` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:37:56.052999 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bb589a-2aa0-4d5e-9a24-b5072f572353 admin admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:37:56.053276 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bb589a-2aa0-4d5e-9a24-b5072f572353 admin admin] PATH_INFO: `/v3/projects/9d989beda87b489c956dc34ab75e5640/users/628214441c41435d8f6f4c1bd0c447d2/roles/8051174e7a7947d5b991ebd793c2e543` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:37:56.068835 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5bb589a-2aa0-4d5e-9a24-b5072f572353 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9d989beda87b489c956dc34ab75e5640, user_id=628214441c41435d8f6f4c1bd0c447d2, role_id=8051174e7a7947d5b991ebd793c2e543)` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:37:56.071701 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5bb589a-2aa0-4d5e-9a24-b5072f572353 admin admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:37:56.085515 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1238/2468] 10.222.0.49 () {60 vars in 1482 bytes} [Wed Jun 15 10:37:56 2022] PUT /identity/v3/projects/9d989beda87b489c956dc34ab75e5640/users/628214441c41435d8f6f4c1bd0c447d2/roles/8051174e7a7947d5b991ebd793c2e543 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:37:56.090227 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dce8f1b-0e3d-4abc-ae73-e848546ad1ab admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:37:56.101108 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dce8f1b-0e3d-4abc-ae73-e848546ad1ab admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:37:56.157555 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5bb589a-2aa0-4d5e-9a24-b5072f572353 admin admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:37:56.159862 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-d5bb589a-2aa0-4d5e-9a24-b5072f572353 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:37:56.162243 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d6188a-f65a-42c3-8b87-a6e3ed920742 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:37:56.163068 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d6188a-f65a-42c3-8b87-a6e3ed920742 admin admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:37:56.163509 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d6188a-f65a-42c3-8b87-a6e3ed920742 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:37:56.164384 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d6188a-f65a-42c3-8b87-a6e3ed920742 admin admin] PATH_INFO: `/v3/roles` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:37:56.164677 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d6188a-f65a-42c3-8b87-a6e3ed920742 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:37:56.166326 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d6188a-f65a-42c3-8b87-a6e3ed920742 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:37:56.173308 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5bb589a-2aa0-4d5e-9a24-b5072f572353 admin admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:37:56.178180 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1231/2469] 10.222.0.49 () {58 vars in 1121 bytes} [Wed Jun 15 10:37:56 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 10:37:56.178367 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e660cf94-2723-4d6c-aae4-34dcf1fbab6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:37:56.182153 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d6188a-f65a-42c3-8b87-a6e3ed920742 admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:37:56.182281 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e660cf94-2723-4d6c-aae4-34dcf1fbab6f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:37:56.182532 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e660cf94-2723-4d6c-aae4-34dcf1fbab6f admin admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:37:56.182818 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e660cf94-2723-4d6c-aae4-34dcf1fbab6f admin admin] PATH_INFO: `/v3/projects/f287efd29e8e411291b802cc5181fa5d` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:37:56.184160 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e660cf94-2723-4d6c-aae4-34dcf1fbab6f admin admin] RBAC: Authorizing `identity:delete_project(project_id=f287efd29e8e411291b802cc5181fa5d)` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:37:56.189048 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e660cf94-2723-4d6c-aae4-34dcf1fbab6f admin admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:37:56.193159 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d6188a-f65a-42c3-8b87-a6e3ed920742 admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:37:56.198066 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1daa4f92-32f9-4995-82fa-dc326c8097b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:37:56.198952 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1daa4f92-32f9-4995-82fa-dc326c8097b2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:37:56.199268 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1daa4f92-32f9-4995-82fa-dc326c8097b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:37:56.199547 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1daa4f92-32f9-4995-82fa-dc326c8097b2 admin admin] PATH_INFO: `/v3/projects/9d989beda87b489c956dc34ab75e5640/users/628214441c41435d8f6f4c1bd0c447d2/roles/8051174e7a7947d5b991ebd793c2e543` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:37:56.201528 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1daa4f92-32f9-4995-82fa-dc326c8097b2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9d989beda87b489c956dc34ab75e5640, user_id=628214441c41435d8f6f4c1bd0c447d2, role_id=8051174e7a7947d5b991ebd793c2e543)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:37:56.204187 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1daa4f92-32f9-4995-82fa-dc326c8097b2 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:37:56.216800 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1232/2470] 10.222.0.49 () {60 vars in 1482 bytes} [Wed Jun 15 10:37:56 2022] PUT /identity/v3/projects/9d989beda87b489c956dc34ab75e5640/users/628214441c41435d8f6f4c1bd0c447d2/roles/8051174e7a7947d5b991ebd793c2e543 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 10:37:56.229701 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0abb7d2-5830-4c1c-8941-c98cbb36a994 None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:37:56.229992 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0abb7d2-5830-4c1c-8941-c98cbb36a994 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:37:56.230267 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0abb7d2-5830-4c1c-8941-c98cbb36a994 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:37:56.237990 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-e660cf94-2723-4d6c-aae4-34dcf1fbab6f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f287efd29e8e411291b802cc5181fa5d', 'request_id': 'req-e660cf94-2723-4d6c-aae4-34dcf1fbab6f'} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:37:56.245768 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-e660cf94-2723-4d6c-aae4-34dcf1fbab6f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f287efd29e8e411291b802cc5181fa5d', 'request_id': 'req-e660cf94-2723-4d6c-aae4-34dcf1fbab6f'} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:37:56.252568 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-e660cf94-2723-4d6c-aae4-34dcf1fbab6f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f287efd29e8e411291b802cc5181fa5d', 'request_id': 'req-e660cf94-2723-4d6c-aae4-34dcf1fbab6f'} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:37:56.256918 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-c0abb7d2-5830-4c1c-8941-c98cbb36a994 None None] MFA Rules not processed for user `628214441c41435d8f6f4c1bd0c447d2`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:37:56.258189 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-c0abb7d2-5830-4c1c-8941-c98cbb36a994 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:37:56.264486 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-e660cf94-2723-4d6c-aae4-34dcf1fbab6f admin admin] The token cache is being invalidate because project f287efd29e8e411291b802cc5181fa5d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80119) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 10:37:56.264774 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-e660cf94-2723-4d6c-aae4-34dcf1fbab6f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:37:56.265420 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-e660cf94-2723-4d6c-aae4-34dcf1fbab6f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:37:56.266999 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1239/2471] 10.222.0.49 () {58 vars in 1232 bytes} [Wed Jun 15 10:37:56 2022] DELETE /identity/v3/projects/f287efd29e8e411291b802cc5181fa5d => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:37:56.271315 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e660cf94-2723-4d6c-aae4-34dcf1fbab6f admin admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:37:56.272724 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-e660cf94-2723-4d6c-aae4-34dcf1fbab6f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:37:56.306586 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:37:56.333321 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1233/2472] 10.222.0.49 () {58 vars in 957 bytes} [Wed Jun 15 10:37:56 2022] POST /identity/v3/auth/tokens => generated 3287 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 10:37:56.340292 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e660cf94-2723-4d6c-aae4-34dcf1fbab6f admin admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:37:56.344285 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0abb7d2-5830-4c1c-8941-c98cbb36a994 None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:37:56.345164 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a7195d-234f-458d-b500-262f1bb45b07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:37:56.345616 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-c0abb7d2-5830-4c1c-8941-c98cbb36a994 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:37:56.346637 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a7195d-234f-458d-b500-262f1bb45b07 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:37:56.346921 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a7195d-234f-458d-b500-262f1bb45b07 admin admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:37:56.347234 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a7195d-234f-458d-b500-262f1bb45b07 admin admin] PATH_INFO: `/v3/users/f5ab6cdc941f4d8d9081c1733fa12abf` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:37:56.348476 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a7195d-234f-458d-b500-262f1bb45b07 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f5ab6cdc941f4d8d9081c1733fa12abf)` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:37:56.350453 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a7195d-234f-458d-b500-262f1bb45b07 admin admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:37:56.381342 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=84,keystone:INSERT=5,keystone:DELETE=11 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:37:56.402096 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-e9a7195d-234f-458d-b500-262f1bb45b07 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f5ab6cdc941f4d8d9081c1733fa12abf'} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:37:56.405226 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0abb7d2-5830-4c1c-8941-c98cbb36a994 None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:37:56.409495 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-e9a7195d-234f-458d-b500-262f1bb45b07 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f5ab6cdc941f4d8d9081c1733fa12abf'} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:37:56.409717 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8215162-71ea-4b07-9c02-9dc48af89d5a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20d1327553404b80aecee87c78fcb52b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '559b3e713f5844b6ae37151d56dd419f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:37:56.410149 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-e9a7195d-234f-458d-b500-262f1bb45b07 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f5ab6cdc941f4d8d9081c1733fa12abf'} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:37:56.410360 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8215162-71ea-4b07-9c02-9dc48af89d5a service cinder] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:37:56.410636 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8215162-71ea-4b07-9c02-9dc48af89d5a service cinder] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:37:56.410850 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-e9a7195d-234f-458d-b500-262f1bb45b07 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f5ab6cdc941f4d8d9081c1733fa12abf'} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:37:56.410948 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8215162-71ea-4b07-9c02-9dc48af89d5a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:37:56.412840 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-a8215162-71ea-4b07-9c02-9dc48af89d5a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:37:56.418829 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-e9a7195d-234f-458d-b500-262f1bb45b07 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f5ab6cdc941f4d8d9081c1733fa12abf'} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:37:56.426547 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8215162-71ea-4b07-9c02-9dc48af89d5a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:37:56.428055 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8215162-71ea-4b07-9c02-9dc48af89d5a service cinder] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:37:56.436068 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1240/2473] 10.222.0.49 () {58 vars in 1223 bytes} [Wed Jun 15 10:37:56 2022] DELETE /identity/v3/users/f5ab6cdc941f4d8d9081c1733fa12abf => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 10:37:56.437619 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1234/2474] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Jun 15 10:37:56 2022] GET /identity/v3/auth/tokens => generated 3287 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 10:37:56.512255 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a7195d-234f-458d-b500-262f1bb45b07 admin admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:37:56.513686 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-e9a7195d-234f-458d-b500-262f1bb45b07 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:37:56.572923 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a7195d-234f-458d-b500-262f1bb45b07 admin admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:37:56.576717 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d469f08-fd9a-42a7-96e9-ecdda5d3da2b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20d1327553404b80aecee87c78fcb52b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '559b3e713f5844b6ae37151d56dd419f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:37:56.577408 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d469f08-fd9a-42a7-96e9-ecdda5d3da2b service cinder] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:37:56.577698 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d469f08-fd9a-42a7-96e9-ecdda5d3da2b service cinder] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:37:56.577982 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d469f08-fd9a-42a7-96e9-ecdda5d3da2b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:37:56.579989 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-4d469f08-fd9a-42a7-96e9-ecdda5d3da2b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:37:56.646498 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d469f08-fd9a-42a7-96e9-ecdda5d3da2b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:37:56.647972 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d469f08-fd9a-42a7-96e9-ecdda5d3da2b service cinder] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:37:56.657412 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1241/2475] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Jun 15 10:37:56 2022] GET /identity/v3/auth/tokens => generated 3222 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:37:56.712562 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8215162-71ea-4b07-9c02-9dc48af89d5a service cinder] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:37:56.714344 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-a8215162-71ea-4b07-9c02-9dc48af89d5a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:37:56.737753 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8215162-71ea-4b07-9c02-9dc48af89d5a service cinder] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:37:56.811593 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca27f3f6-22f3-4af5-8bdf-8ef9b9606f1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:37:56.812548 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca27f3f6-22f3-4af5-8bdf-8ef9b9606f1c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:37:56.812832 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca27f3f6-22f3-4af5-8bdf-8ef9b9606f1c admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:37:56.813098 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca27f3f6-22f3-4af5-8bdf-8ef9b9606f1c admin admin] PATH_INFO: `/v3/projects/4dde1515c2ac4e38a76b81aeaa2347e8` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:37:56.814368 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca27f3f6-22f3-4af5-8bdf-8ef9b9606f1c admin admin] RBAC: Authorizing `identity:delete_project(project_id=4dde1515c2ac4e38a76b81aeaa2347e8)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:37:56.816327 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca27f3f6-22f3-4af5-8bdf-8ef9b9606f1c admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:37:56.864388 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-ca27f3f6-22f3-4af5-8bdf-8ef9b9606f1c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4dde1515c2ac4e38a76b81aeaa2347e8', 'request_id': 'req-ca27f3f6-22f3-4af5-8bdf-8ef9b9606f1c'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:37:56.871502 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-ca27f3f6-22f3-4af5-8bdf-8ef9b9606f1c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4dde1515c2ac4e38a76b81aeaa2347e8', 'request_id': 'req-ca27f3f6-22f3-4af5-8bdf-8ef9b9606f1c'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:37:56.877131 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-ca27f3f6-22f3-4af5-8bdf-8ef9b9606f1c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4dde1515c2ac4e38a76b81aeaa2347e8', 'request_id': 'req-ca27f3f6-22f3-4af5-8bdf-8ef9b9606f1c'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:37:56.887971 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-ca27f3f6-22f3-4af5-8bdf-8ef9b9606f1c admin admin] The token cache is being invalidate because project 4dde1515c2ac4e38a76b81aeaa2347e8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80120) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 10:37:56.888260 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-ca27f3f6-22f3-4af5-8bdf-8ef9b9606f1c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:37:56.888933 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-ca27f3f6-22f3-4af5-8bdf-8ef9b9606f1c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:37:56.890467 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1235/2476] 10.222.0.49 () {58 vars in 1232 bytes} [Wed Jun 15 10:37:56 2022] DELETE /identity/v3/projects/4dde1515c2ac4e38a76b81aeaa2347e8 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:37:56.896856 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b0fbb7-e92e-43b7-991a-7b0c9dd841a7 None None] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:37:56.897147 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b0fbb7-e92e-43b7-991a-7b0c9dd841a7 None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:37:56.897426 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b0fbb7-e92e-43b7-991a-7b0c9dd841a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:37:56.917291 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.auth.core [None req-a6b0fbb7-e92e-43b7-991a-7b0c9dd841a7 None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:37:56.918595 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-a6b0fbb7-e92e-43b7-991a-7b0c9dd841a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:37:56.968069 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:DELETE=6,keystone:SELECT=93,keystone:INSERT=1 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:37:56.979753 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1242/2477] 10.222.0.49 () {58 vars in 957 bytes} [Wed Jun 15 10:37:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:37:56.987639 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2f069f-921d-4e5d-a0d4-f3a0bf771582 None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:37:56.987927 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2f069f-921d-4e5d-a0d4-f3a0bf771582 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:37:56.988231 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2f069f-921d-4e5d-a0d4-f3a0bf771582 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:37:57.006324 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-bb2f069f-921d-4e5d-a0d4-f3a0bf771582 None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:37:57.007549 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-bb2f069f-921d-4e5d-a0d4-f3a0bf771582 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:37:57.015324 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1236/2478] 10.222.0.49 () {58 vars in 957 bytes} [Wed Jun 15 10:37:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:37:57.020099 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b0fbb7-e92e-43b7-991a-7b0c9dd841a7 None None] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:37:57.030055 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b0fbb7-e92e-43b7-991a-7b0c9dd841a7 None None] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:37:57.035272 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4efa874-fbc8-4a2a-ab95-33fad9fd3025 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:37:57.035925 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4efa874-fbc8-4a2a-ab95-33fad9fd3025 admin admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:37:57.036210 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4efa874-fbc8-4a2a-ab95-33fad9fd3025 admin admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:37:57.036488 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4efa874-fbc8-4a2a-ab95-33fad9fd3025 admin admin] PATH_INFO: `/v3/domains` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:37:57.037235 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4efa874-fbc8-4a2a-ab95-33fad9fd3025 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:37:57.037540 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4efa874-fbc8-4a2a-ab95-33fad9fd3025 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:37:57.039178 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4efa874-fbc8-4a2a-ab95-33fad9fd3025 admin admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:37:57.050567 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1243/2479] 10.222.0.49 () {58 vars in 1152 bytes} [Wed Jun 15 10:37:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:37:57.054687 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2f069f-921d-4e5d-a0d4-f3a0bf771582 None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:37:57.063819 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2f069f-921d-4e5d-a0d4-f3a0bf771582 None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:37:57.068358 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84389ff-5137-45c4-b147-cb4ca87114db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:37:57.068998 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84389ff-5137-45c4-b147-cb4ca87114db admin admin] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:37:57.069279 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84389ff-5137-45c4-b147-cb4ca87114db admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:37:57.069552 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84389ff-5137-45c4-b147-cb4ca87114db admin admin] PATH_INFO: `/v3/projects` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:37:57.070237 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e84389ff-5137-45c4-b147-cb4ca87114db admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:37:57.072168 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e84389ff-5137-45c4-b147-cb4ca87114db admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:37:57.075303 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:37:57.100371 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1237/2480] 10.222.0.49 () {60 vars in 1152 bytes} [Wed Jun 15 10:37:57 2022] POST /identity/v3/projects => generated 365 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 10:37:57.103903 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4efa874-fbc8-4a2a-ab95-33fad9fd3025 admin admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:37:57.113657 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4efa874-fbc8-4a2a-ab95-33fad9fd3025 admin admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:37:57.168105 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6f5fa5-61ce-4148-9e8b-7679db3af150 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:37:57.168698 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6f5fa5-61ce-4148-9e8b-7679db3af150 admin admin] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:37:57.168962 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6f5fa5-61ce-4148-9e8b-7679db3af150 admin admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:37:57.169236 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6f5fa5-61ce-4148-9e8b-7679db3af150 admin admin] PATH_INFO: `/v3/users` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:37:57.169916 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e6f5fa5-61ce-4148-9e8b-7679db3af150 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:37:57.171853 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e6f5fa5-61ce-4148-9e8b-7679db3af150 admin admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:37:57.192334 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1244/2481] 10.222.0.49 () {60 vars in 1143 bytes} [Wed Jun 15 10:37:57 2022] POST /identity/v3/users => generated 283 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 10:37:57.196625 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84389ff-5137-45c4-b147-cb4ca87114db admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:37:57.207329 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84389ff-5137-45c4-b147-cb4ca87114db admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:37:57.211944 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4420d287-0c76-4b59-8ea4-21ac256578b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:37:57.212625 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4420d287-0c76-4b59-8ea4-21ac256578b9 admin admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:37:57.212910 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4420d287-0c76-4b59-8ea4-21ac256578b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:37:57.213187 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4420d287-0c76-4b59-8ea4-21ac256578b9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:37:57.213912 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4420d287-0c76-4b59-8ea4-21ac256578b9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:37:57.215577 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4420d287-0c76-4b59-8ea4-21ac256578b9 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:37:57.228503 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1238/2482] 10.222.0.49 () {58 vars in 1121 bytes} [Wed Jun 15 10:37:57 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 10:37:57.229571 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6f5fa5-61ce-4148-9e8b-7679db3af150 admin admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:37:57.239488 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6f5fa5-61ce-4148-9e8b-7679db3af150 admin admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:37:57.244094 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369cbc7f-2154-48b6-9aff-e5341c88e763 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:37:57.244752 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369cbc7f-2154-48b6-9aff-e5341c88e763 admin admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:37:57.245037 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369cbc7f-2154-48b6-9aff-e5341c88e763 admin admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:37:57.245318 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369cbc7f-2154-48b6-9aff-e5341c88e763 admin admin] PATH_INFO: `/v3/roles` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:37:57.246036 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-369cbc7f-2154-48b6-9aff-e5341c88e763 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:37:57.247713 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-369cbc7f-2154-48b6-9aff-e5341c88e763 admin admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:37:57.258019 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1245/2483] 10.222.0.49 () {58 vars in 1121 bytes} [Wed Jun 15 10:37:57 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 10:37:57.262171 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4420d287-0c76-4b59-8ea4-21ac256578b9 admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:37:57.272474 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4420d287-0c76-4b59-8ea4-21ac256578b9 admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:37:57.277297 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e6070d-18dd-495f-97a2-8e074a04fff8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:37:57.278177 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e6070d-18dd-495f-97a2-8e074a04fff8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:37:57.278444 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e6070d-18dd-495f-97a2-8e074a04fff8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:37:57.278718 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e6070d-18dd-495f-97a2-8e074a04fff8 admin admin] PATH_INFO: `/v3/projects/26798c32faa147409b464abe67438e4b/users/80e35b835c1a4289956be479b35e3a31/roles/6876e57cc51c448097ab109d35b2a3da` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:37:57.293691 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e6070d-18dd-495f-97a2-8e074a04fff8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=26798c32faa147409b464abe67438e4b, user_id=80e35b835c1a4289956be479b35e3a31, role_id=6876e57cc51c448097ab109d35b2a3da)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:37:57.296396 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e6070d-18dd-495f-97a2-8e074a04fff8 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:37:57.308847 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1239/2484] 10.222.0.49 () {60 vars in 1482 bytes} [Wed Jun 15 10:37:57 2022] PUT /identity/v3/projects/26798c32faa147409b464abe67438e4b/users/80e35b835c1a4289956be479b35e3a31/roles/6876e57cc51c448097ab109d35b2a3da => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:37:57.370944 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82269ffd-c07b-4229-b18b-44c2f885ac05 None None] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:37:57.371270 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82269ffd-c07b-4229-b18b-44c2f885ac05 None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:37:57.371553 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82269ffd-c07b-4229-b18b-44c2f885ac05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:37:57.398798 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.auth.core [None req-82269ffd-c07b-4229-b18b-44c2f885ac05 None None] MFA Rules not processed for user `80e35b835c1a4289956be479b35e3a31`. Rule list: `[]` (Enabled: `True`). {{(pid=80119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:37:57.400310 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-82269ffd-c07b-4229-b18b-44c2f885ac05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:37:57.461966 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1246/2485] 10.222.0.49 () {58 vars in 957 bytes} [Wed Jun 15 10:37:57 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:37:57.473696 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e6070d-18dd-495f-97a2-8e074a04fff8 admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:37:57.475419 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-94e6070d-18dd-495f-97a2-8e074a04fff8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:37:57.531298 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e6070d-18dd-495f-97a2-8e074a04fff8 admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:37:57.537439 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b3f98f-b2fa-4849-b08c-268eb5984f38 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20d1327553404b80aecee87c78fcb52b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '559b3e713f5844b6ae37151d56dd419f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:37:57.538083 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b3f98f-b2fa-4849-b08c-268eb5984f38 service cinder] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:37:57.538365 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b3f98f-b2fa-4849-b08c-268eb5984f38 service cinder] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:37:57.538640 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b3f98f-b2fa-4849-b08c-268eb5984f38 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:37:57.552934 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b3f98f-b2fa-4849-b08c-268eb5984f38 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:37:57.555269 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b3f98f-b2fa-4849-b08c-268eb5984f38 service cinder] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:37:57.564168 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1240/2486] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Jun 15 10:37:57 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:38:07.459015 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=73,keystone:INSERT=3 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:38:07.560130 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=47,keystone:INSERT=2 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:38:13.548141 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82269ffd-c07b-4229-b18b-44c2f885ac05 None None] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:38:13.550149 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-82269ffd-c07b-4229-b18b-44c2f885ac05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:38:13.614835 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82269ffd-c07b-4229-b18b-44c2f885ac05 None None] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:38:13.619702 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4864176b-009f-4ee5-b6a2-dbf895ede62b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:38:13.620659 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4864176b-009f-4ee5-b6a2-dbf895ede62b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:38:13.620928 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4864176b-009f-4ee5-b6a2-dbf895ede62b admin admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:38:13.621209 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4864176b-009f-4ee5-b6a2-dbf895ede62b admin admin] PATH_INFO: `/v3/users/205068af1e284d7fb7f0d4a93b0cdb3c` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:38:13.622450 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4864176b-009f-4ee5-b6a2-dbf895ede62b admin admin] RBAC: Authorizing `identity:delete_user(user_id=205068af1e284d7fb7f0d4a93b0cdb3c)` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:38:13.624414 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4864176b-009f-4ee5-b6a2-dbf895ede62b admin admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:38:13.675666 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-4864176b-009f-4ee5-b6a2-dbf895ede62b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '205068af1e284d7fb7f0d4a93b0cdb3c'} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:38:13.682529 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-4864176b-009f-4ee5-b6a2-dbf895ede62b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '205068af1e284d7fb7f0d4a93b0cdb3c'} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:38:13.683229 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-4864176b-009f-4ee5-b6a2-dbf895ede62b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '205068af1e284d7fb7f0d4a93b0cdb3c'} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:38:13.683844 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-4864176b-009f-4ee5-b6a2-dbf895ede62b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '205068af1e284d7fb7f0d4a93b0cdb3c'} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:38:13.692375 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-4864176b-009f-4ee5-b6a2-dbf895ede62b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '205068af1e284d7fb7f0d4a93b0cdb3c'} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:38:13.709984 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1247/2487] 10.222.0.49 () {58 vars in 1223 bytes} [Wed Jun 15 10:38:13 2022] DELETE /identity/v3/users/205068af1e284d7fb7f0d4a93b0cdb3c => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:38:13.714260 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abdc577-81c1-46f1-96da-40eaebfe5e19 None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:38:13.714720 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abdc577-81c1-46f1-96da-40eaebfe5e19 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:38:13.714816 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abdc577-81c1-46f1-96da-40eaebfe5e19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:38:13.733220 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-4abdc577-81c1-46f1-96da-40eaebfe5e19 None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:38:13.733532 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-4abdc577-81c1-46f1-96da-40eaebfe5e19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:38:13.790999 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1241/2488] 10.222.0.49 () {58 vars in 957 bytes} [Wed Jun 15 10:38:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 10:38:13.801372 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4864176b-009f-4ee5-b6a2-dbf895ede62b admin admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:38:13.803368 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-4864176b-009f-4ee5-b6a2-dbf895ede62b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:38:13.860165 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4864176b-009f-4ee5-b6a2-dbf895ede62b admin admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:38:13.864303 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cbeb122-bccd-4d62-a84a-69ee1f09b8ca service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0cb44a17d205431793e9a8791e477a32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '559b3e713f5844b6ae37151d56dd419f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:38:13.864943 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbeb122-bccd-4d62-a84a-69ee1f09b8ca service neutron] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:38:13.865224 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbeb122-bccd-4d62-a84a-69ee1f09b8ca service neutron] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:38:13.865495 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbeb122-bccd-4d62-a84a-69ee1f09b8ca service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:38:13.874224 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cbeb122-bccd-4d62-a84a-69ee1f09b8ca service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:38:13.875679 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cbeb122-bccd-4d62-a84a-69ee1f09b8ca service neutron] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:38:13.884903 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1248/2489] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Jun 15 10:38:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 10:38:14.165991 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4abdc577-81c1-46f1-96da-40eaebfe5e19 None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:38:14.167608 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-4abdc577-81c1-46f1-96da-40eaebfe5e19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:38:14.187931 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4abdc577-81c1-46f1-96da-40eaebfe5e19 None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:38:14.196605 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35fc462e-11e8-416c-b492-811dbc3d68fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:38:14.197900 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fc462e-11e8-416c-b492-811dbc3d68fa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:38:14.198480 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fc462e-11e8-416c-b492-811dbc3d68fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:38:14.198756 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fc462e-11e8-416c-b492-811dbc3d68fa admin admin] PATH_INFO: `/v3/projects/47ef8e14002f446f98ae1c5ac1b11370` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:38:14.200146 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35fc462e-11e8-416c-b492-811dbc3d68fa admin admin] RBAC: Authorizing `identity:delete_project(project_id=47ef8e14002f446f98ae1c5ac1b11370)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:38:14.202062 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35fc462e-11e8-416c-b492-811dbc3d68fa admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:38:14.254513 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-35fc462e-11e8-416c-b492-811dbc3d68fa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '47ef8e14002f446f98ae1c5ac1b11370', 'request_id': 'req-35fc462e-11e8-416c-b492-811dbc3d68fa'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:38:14.263708 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-35fc462e-11e8-416c-b492-811dbc3d68fa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '47ef8e14002f446f98ae1c5ac1b11370', 'request_id': 'req-35fc462e-11e8-416c-b492-811dbc3d68fa'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:38:14.272098 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-35fc462e-11e8-416c-b492-811dbc3d68fa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '47ef8e14002f446f98ae1c5ac1b11370', 'request_id': 'req-35fc462e-11e8-416c-b492-811dbc3d68fa'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:38:14.284092 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-35fc462e-11e8-416c-b492-811dbc3d68fa admin admin] The token cache is being invalidate because project 47ef8e14002f446f98ae1c5ac1b11370 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80120) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 10:38:14.284380 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-35fc462e-11e8-416c-b492-811dbc3d68fa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:38:14.285034 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-35fc462e-11e8-416c-b492-811dbc3d68fa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:38:14.286668 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1242/2490] 10.222.0.49 () {58 vars in 1232 bytes} [Wed Jun 15 10:38:14 2022] DELETE /identity/v3/projects/47ef8e14002f446f98ae1c5ac1b11370 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:38:14.290883 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cbeb122-bccd-4d62-a84a-69ee1f09b8ca service neutron] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:38:14.302123 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cbeb122-bccd-4d62-a84a-69ee1f09b8ca service neutron] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:38:14.335471 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:38:14.370094 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f821b23-5e80-415c-bac5-7d51ea4b1209 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:38:14.370975 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f821b23-5e80-415c-bac5-7d51ea4b1209 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:38:14.371288 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f821b23-5e80-415c-bac5-7d51ea4b1209 admin admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:38:14.371568 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f821b23-5e80-415c-bac5-7d51ea4b1209 admin admin] PATH_INFO: `/v3/users/628214441c41435d8f6f4c1bd0c447d2` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:38:14.372775 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f821b23-5e80-415c-bac5-7d51ea4b1209 admin admin] RBAC: Authorizing `identity:delete_user(user_id=628214441c41435d8f6f4c1bd0c447d2)` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:38:14.374690 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f821b23-5e80-415c-bac5-7d51ea4b1209 admin admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:38:14.421637 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-7f821b23-5e80-415c-bac5-7d51ea4b1209 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '628214441c41435d8f6f4c1bd0c447d2'} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:38:14.428367 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-7f821b23-5e80-415c-bac5-7d51ea4b1209 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '628214441c41435d8f6f4c1bd0c447d2'} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:38:14.429022 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-7f821b23-5e80-415c-bac5-7d51ea4b1209 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '628214441c41435d8f6f4c1bd0c447d2'} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:38:14.429630 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-7f821b23-5e80-415c-bac5-7d51ea4b1209 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '628214441c41435d8f6f4c1bd0c447d2'} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:38:14.436444 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-7f821b23-5e80-415c-bac5-7d51ea4b1209 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '628214441c41435d8f6f4c1bd0c447d2'} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:38:14.454366 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1249/2491] 10.222.0.49 () {58 vars in 1223 bytes} [Wed Jun 15 10:38:14 2022] DELETE /identity/v3/users/628214441c41435d8f6f4c1bd0c447d2 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:38:14.726491 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35fc462e-11e8-416c-b492-811dbc3d68fa admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:38:14.731694 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-35fc462e-11e8-416c-b492-811dbc3d68fa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:38:14.746159 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35fc462e-11e8-416c-b492-811dbc3d68fa admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:38:14.814488 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967f2afa-450c-42b7-9a79-a2f476311b50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:38:14.815384 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967f2afa-450c-42b7-9a79-a2f476311b50 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:38:14.815665 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967f2afa-450c-42b7-9a79-a2f476311b50 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:38:14.815937 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967f2afa-450c-42b7-9a79-a2f476311b50 admin admin] PATH_INFO: `/v3/projects/9d989beda87b489c956dc34ab75e5640` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:38:14.817104 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-967f2afa-450c-42b7-9a79-a2f476311b50 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9d989beda87b489c956dc34ab75e5640)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:38:14.818988 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-967f2afa-450c-42b7-9a79-a2f476311b50 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:38:14.855217 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:38:14.875377 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-967f2afa-450c-42b7-9a79-a2f476311b50 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9d989beda87b489c956dc34ab75e5640', 'request_id': 'req-967f2afa-450c-42b7-9a79-a2f476311b50'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:38:14.883025 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-967f2afa-450c-42b7-9a79-a2f476311b50 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9d989beda87b489c956dc34ab75e5640', 'request_id': 'req-967f2afa-450c-42b7-9a79-a2f476311b50'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:38:14.889400 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-967f2afa-450c-42b7-9a79-a2f476311b50 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9d989beda87b489c956dc34ab75e5640', 'request_id': 'req-967f2afa-450c-42b7-9a79-a2f476311b50'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:38:14.901881 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-967f2afa-450c-42b7-9a79-a2f476311b50 admin admin] The token cache is being invalidate because project 9d989beda87b489c956dc34ab75e5640 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80120) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 10:38:14.902175 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-967f2afa-450c-42b7-9a79-a2f476311b50 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:38:14.902852 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-967f2afa-450c-42b7-9a79-a2f476311b50 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:38:14.904434 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1243/2492] 10.222.0.49 () {58 vars in 1232 bytes} [Wed Jun 15 10:38:14 2022] DELETE /identity/v3/projects/9d989beda87b489c956dc34ab75e5640 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:38:14.910042 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb41848-bbd5-4f14-8fdc-951e53fefaa3 None None] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:38:14.910512 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb41848-bbd5-4f14-8fdc-951e53fefaa3 None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:38:14.910608 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb41848-bbd5-4f14-8fdc-951e53fefaa3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:38:14.930285 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.auth.core [None req-5fb41848-bbd5-4f14-8fdc-951e53fefaa3 None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:38:14.931528 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-5fb41848-bbd5-4f14-8fdc-951e53fefaa3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:38:14.988360 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1250/2493] 10.222.0.49 () {58 vars in 957 bytes} [Wed Jun 15 10:38:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:38:14.996171 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee9cf77-49d3-4f44-a42c-72965179f783 None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:38:14.996459 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee9cf77-49d3-4f44-a42c-72965179f783 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:38:14.996734 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee9cf77-49d3-4f44-a42c-72965179f783 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:38:15.014406 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-bee9cf77-49d3-4f44-a42c-72965179f783 None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:38:15.015638 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-bee9cf77-49d3-4f44-a42c-72965179f783 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:38:15.023994 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1244/2494] 10.222.0.49 () {58 vars in 957 bytes} [Wed Jun 15 10:38:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:38:15.028483 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb41848-bbd5-4f14-8fdc-951e53fefaa3 None None] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:38:15.038684 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb41848-bbd5-4f14-8fdc-951e53fefaa3 None None] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:38:15.043244 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59fa05aa-1f34-4671-83fd-ebdeb6e24707 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:38:15.043896 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fa05aa-1f34-4671-83fd-ebdeb6e24707 admin admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:38:15.044176 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fa05aa-1f34-4671-83fd-ebdeb6e24707 admin admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:38:15.044449 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fa05aa-1f34-4671-83fd-ebdeb6e24707 admin admin] PATH_INFO: `/v3/domains` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:38:15.045163 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59fa05aa-1f34-4671-83fd-ebdeb6e24707 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:38:15.045493 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59fa05aa-1f34-4671-83fd-ebdeb6e24707 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:38:15.047114 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59fa05aa-1f34-4671-83fd-ebdeb6e24707 admin admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:38:15.058438 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1251/2495] 10.222.0.49 () {58 vars in 1152 bytes} [Wed Jun 15 10:38:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 10:38:15.062731 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee9cf77-49d3-4f44-a42c-72965179f783 None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:38:15.071963 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee9cf77-49d3-4f44-a42c-72965179f783 None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:38:15.076670 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6463dc11-edf3-46ed-802e-3c010cc18a47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:38:15.077324 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6463dc11-edf3-46ed-802e-3c010cc18a47 admin admin] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:38:15.077605 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6463dc11-edf3-46ed-802e-3c010cc18a47 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:38:15.077882 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6463dc11-edf3-46ed-802e-3c010cc18a47 admin admin] PATH_INFO: `/v3/projects` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:38:15.078572 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6463dc11-edf3-46ed-802e-3c010cc18a47 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:38:15.081012 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6463dc11-edf3-46ed-802e-3c010cc18a47 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:38:15.116551 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1245/2496] 10.222.0.49 () {60 vars in 1152 bytes} [Wed Jun 15 10:38:15 2022] POST /identity/v3/projects => generated 365 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 10:38:15.117599 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59fa05aa-1f34-4671-83fd-ebdeb6e24707 admin admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:38:15.127503 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59fa05aa-1f34-4671-83fd-ebdeb6e24707 admin admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:38:15.154122 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:38:15.189964 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0450cd02-bb31-4b2f-9523-1b03040c396c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:38:15.190662 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0450cd02-bb31-4b2f-9523-1b03040c396c admin admin] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:38:15.190944 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0450cd02-bb31-4b2f-9523-1b03040c396c admin admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:38:15.191254 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0450cd02-bb31-4b2f-9523-1b03040c396c admin admin] PATH_INFO: `/v3/users` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:38:15.191933 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0450cd02-bb31-4b2f-9523-1b03040c396c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:38:15.193857 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0450cd02-bb31-4b2f-9523-1b03040c396c admin admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:38:15.216669 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1252/2497] 10.222.0.49 () {60 vars in 1143 bytes} [Wed Jun 15 10:38:15 2022] POST /identity/v3/users => generated 283 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 10:38:15.221198 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6463dc11-edf3-46ed-802e-3c010cc18a47 admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:38:15.232881 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6463dc11-edf3-46ed-802e-3c010cc18a47 admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:38:15.238953 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b5f203-504a-4e6d-9dc4-9a11c02f1a90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:38:15.239822 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b5f203-504a-4e6d-9dc4-9a11c02f1a90 admin admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:38:15.240112 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b5f203-504a-4e6d-9dc4-9a11c02f1a90 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:38:15.240395 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b5f203-504a-4e6d-9dc4-9a11c02f1a90 admin admin] PATH_INFO: `/v3/roles` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:38:15.241394 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b5f203-504a-4e6d-9dc4-9a11c02f1a90 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:38:15.243365 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b5f203-504a-4e6d-9dc4-9a11c02f1a90 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:38:15.256177 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1246/2498] 10.222.0.49 () {58 vars in 1121 bytes} [Wed Jun 15 10:38:15 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 10:38:15.260056 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0450cd02-bb31-4b2f-9523-1b03040c396c admin admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:38:15.271006 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0450cd02-bb31-4b2f-9523-1b03040c396c admin admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:38:15.275889 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e388aecc-0041-4631-9dbb-edb32fd3331e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:38:15.276553 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e388aecc-0041-4631-9dbb-edb32fd3331e admin admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:38:15.276839 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e388aecc-0041-4631-9dbb-edb32fd3331e admin admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:38:15.277118 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e388aecc-0041-4631-9dbb-edb32fd3331e admin admin] PATH_INFO: `/v3/roles` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:38:15.277840 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e388aecc-0041-4631-9dbb-edb32fd3331e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:38:15.279494 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e388aecc-0041-4631-9dbb-edb32fd3331e admin admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:38:15.289347 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1253/2499] 10.222.0.49 () {58 vars in 1121 bytes} [Wed Jun 15 10:38:15 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 10:38:15.293452 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b5f203-504a-4e6d-9dc4-9a11c02f1a90 admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:38:15.303363 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b5f203-504a-4e6d-9dc4-9a11c02f1a90 admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:38:15.307910 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26578b34-f2ac-47af-bc2c-2f35554ffb66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:38:15.308776 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26578b34-f2ac-47af-bc2c-2f35554ffb66 admin admin] REQUEST_METHOD: `PUT` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:38:15.309057 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26578b34-f2ac-47af-bc2c-2f35554ffb66 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:38:15.309325 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26578b34-f2ac-47af-bc2c-2f35554ffb66 admin admin] PATH_INFO: `/v3/projects/17baaaf947ab4425932593910cad8472/users/d96182369bf7483a855a6daa2dde3cba/roles/6876e57cc51c448097ab109d35b2a3da` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:38:15.327034 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26578b34-f2ac-47af-bc2c-2f35554ffb66 admin admin] RBAC: Authorizing `identity:create_grant(project_id=17baaaf947ab4425932593910cad8472, user_id=d96182369bf7483a855a6daa2dde3cba, role_id=6876e57cc51c448097ab109d35b2a3da)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:38:15.330407 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26578b34-f2ac-47af-bc2c-2f35554ffb66 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:38:15.346111 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1247/2500] 10.222.0.49 () {60 vars in 1482 bytes} [Wed Jun 15 10:38:15 2022] PUT /identity/v3/projects/17baaaf947ab4425932593910cad8472/users/d96182369bf7483a855a6daa2dde3cba/roles/6876e57cc51c448097ab109d35b2a3da => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:38:15.362310 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61292330-8086-409c-82f6-32d2783e9915 None None] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:38:15.362310 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61292330-8086-409c-82f6-32d2783e9915 None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:38:15.362310 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61292330-8086-409c-82f6-32d2783e9915 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:38:15.390302 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.auth.core [None req-61292330-8086-409c-82f6-32d2783e9915 None None] MFA Rules not processed for user `d96182369bf7483a855a6daa2dde3cba`. Rule list: `[]` (Enabled: `True`). {{(pid=80119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:38:15.391896 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-61292330-8086-409c-82f6-32d2783e9915 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:38:15.459358 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1254/2501] 10.222.0.49 () {58 vars in 957 bytes} [Wed Jun 15 10:38:15 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:38:15.468256 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26578b34-f2ac-47af-bc2c-2f35554ffb66 admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:38:15.532025 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26578b34-f2ac-47af-bc2c-2f35554ffb66 admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:38:15.536713 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b82fa4-a64d-4e74-8c92-212f68dece32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:38:15.537373 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b82fa4-a64d-4e74-8c92-212f68dece32 admin admin] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:38:15.537656 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b82fa4-a64d-4e74-8c92-212f68dece32 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:38:15.537933 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b82fa4-a64d-4e74-8c92-212f68dece32 admin admin] PATH_INFO: `/v3/projects` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:38:15.538622 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b82fa4-a64d-4e74-8c92-212f68dece32 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:38:15.540536 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b82fa4-a64d-4e74-8c92-212f68dece32 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:38:15.584164 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1248/2502] 10.222.0.49 () {60 vars in 1152 bytes} [Wed Jun 15 10:38:15 2022] POST /identity/v3/projects => generated 365 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 10:38:15.588354 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61292330-8086-409c-82f6-32d2783e9915 None None] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:38:15.600036 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61292330-8086-409c-82f6-32d2783e9915 None None] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:38:15.661067 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d82fda3-0616-4318-95dc-0caf78ab8eeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:38:15.661818 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d82fda3-0616-4318-95dc-0caf78ab8eeb admin admin] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:38:15.662178 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d82fda3-0616-4318-95dc-0caf78ab8eeb admin admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:38:15.662526 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d82fda3-0616-4318-95dc-0caf78ab8eeb admin admin] PATH_INFO: `/v3/users` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:38:15.663321 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d82fda3-0616-4318-95dc-0caf78ab8eeb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:38:15.665253 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d82fda3-0616-4318-95dc-0caf78ab8eeb admin admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:38:15.695644 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1255/2503] 10.222.0.49 () {60 vars in 1143 bytes} [Wed Jun 15 10:38:15 2022] POST /identity/v3/users => generated 289 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 10:38:15.697285 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b82fa4-a64d-4e74-8c92-212f68dece32 admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:38:15.709082 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b82fa4-a64d-4e74-8c92-212f68dece32 admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:38:15.714970 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e68e6e-f15f-411e-a2d5-c7b2a9c27ad2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:38:15.715810 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e68e6e-f15f-411e-a2d5-c7b2a9c27ad2 admin admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:38:15.716221 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e68e6e-f15f-411e-a2d5-c7b2a9c27ad2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:38:15.716593 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e68e6e-f15f-411e-a2d5-c7b2a9c27ad2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:38:15.717458 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e68e6e-f15f-411e-a2d5-c7b2a9c27ad2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:38:15.719289 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e68e6e-f15f-411e-a2d5-c7b2a9c27ad2 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:38:15.734238 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1249/2504] 10.222.0.49 () {58 vars in 1121 bytes} [Wed Jun 15 10:38:15 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 10:38:15.736706 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d82fda3-0616-4318-95dc-0caf78ab8eeb admin admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:38:15.746706 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d82fda3-0616-4318-95dc-0caf78ab8eeb admin admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:38:15.752057 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b0bf0b-bd8b-40d5-9c61-2031a07dc7e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:38:15.752951 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b0bf0b-bd8b-40d5-9c61-2031a07dc7e4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:38:15.753267 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b0bf0b-bd8b-40d5-9c61-2031a07dc7e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:38:15.753545 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b0bf0b-bd8b-40d5-9c61-2031a07dc7e4 admin admin] PATH_INFO: `/v3/projects/3a63f625c6274fce8a275e3bad15ce78/users/a5afa4f0fc7f42cc8df65e637dd0e077/roles/8051174e7a7947d5b991ebd793c2e543` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:38:15.772921 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b0bf0b-bd8b-40d5-9c61-2031a07dc7e4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3a63f625c6274fce8a275e3bad15ce78, user_id=a5afa4f0fc7f42cc8df65e637dd0e077, role_id=8051174e7a7947d5b991ebd793c2e543)` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:38:15.776057 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b0bf0b-bd8b-40d5-9c61-2031a07dc7e4 admin admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:38:15.791806 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1256/2505] 10.222.0.49 () {60 vars in 1482 bytes} [Wed Jun 15 10:38:15 2022] PUT /identity/v3/projects/3a63f625c6274fce8a275e3bad15ce78/users/a5afa4f0fc7f42cc8df65e637dd0e077/roles/8051174e7a7947d5b991ebd793c2e543 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:38:15.797071 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e68e6e-f15f-411e-a2d5-c7b2a9c27ad2 admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:38:15.810035 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e68e6e-f15f-411e-a2d5-c7b2a9c27ad2 admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:38:15.849011 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:38:15.893007 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24047150-ce82-4c30-ba7f-158f2628a49e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:38:15.893817 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24047150-ce82-4c30-ba7f-158f2628a49e admin admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:38:15.894188 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24047150-ce82-4c30-ba7f-158f2628a49e admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:38:15.894743 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24047150-ce82-4c30-ba7f-158f2628a49e admin admin] PATH_INFO: `/v3/roles` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:38:15.895700 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24047150-ce82-4c30-ba7f-158f2628a49e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:38:15.897867 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24047150-ce82-4c30-ba7f-158f2628a49e admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:38:15.912365 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1250/2506] 10.222.0.49 () {58 vars in 1121 bytes} [Wed Jun 15 10:38:15 2022] GET /identity/v3/roles => generated 1420 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 10:38:15.913606 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b0bf0b-bd8b-40d5-9c61-2031a07dc7e4 admin admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:38:15.919102 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:38:15.927421 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b0bf0b-bd8b-40d5-9c61-2031a07dc7e4 admin admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:38:15.932394 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e319fe2-14ba-4dbc-9604-d163456e1d21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:38:15.933386 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e319fe2-14ba-4dbc-9604-d163456e1d21 admin admin] REQUEST_METHOD: `PUT` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:38:15.933676 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e319fe2-14ba-4dbc-9604-d163456e1d21 admin admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:38:15.933957 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e319fe2-14ba-4dbc-9604-d163456e1d21 admin admin] PATH_INFO: `/v3/projects/3a63f625c6274fce8a275e3bad15ce78/users/a5afa4f0fc7f42cc8df65e637dd0e077/roles/8051174e7a7947d5b991ebd793c2e543` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:38:15.935999 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e319fe2-14ba-4dbc-9604-d163456e1d21 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3a63f625c6274fce8a275e3bad15ce78, user_id=a5afa4f0fc7f42cc8df65e637dd0e077, role_id=8051174e7a7947d5b991ebd793c2e543)` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:38:15.938656 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e319fe2-14ba-4dbc-9604-d163456e1d21 admin admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:38:15.949649 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1257/2507] 10.222.0.49 () {60 vars in 1482 bytes} [Wed Jun 15 10:38:15 2022] PUT /identity/v3/projects/3a63f625c6274fce8a275e3bad15ce78/users/a5afa4f0fc7f42cc8df65e637dd0e077/roles/8051174e7a7947d5b991ebd793c2e543 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:38:15.960594 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1f137a-1812-42a2-a651-c6d8e9362ab2 None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:38:15.963280 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1f137a-1812-42a2-a651-c6d8e9362ab2 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:38:15.964487 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1f137a-1812-42a2-a651-c6d8e9362ab2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:38:16.000476 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-6c1f137a-1812-42a2-a651-c6d8e9362ab2 None None] MFA Rules not processed for user `a5afa4f0fc7f42cc8df65e637dd0e077`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:38:16.001745 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-6c1f137a-1812-42a2-a651-c6d8e9362ab2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:38:16.069189 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1251/2508] 10.222.0.49 () {58 vars in 957 bytes} [Wed Jun 15 10:38:15 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:38:16.080758 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e319fe2-14ba-4dbc-9604-d163456e1d21 admin admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:38:16.082589 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-7e319fe2-14ba-4dbc-9604-d163456e1d21 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:38:16.145468 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e319fe2-14ba-4dbc-9604-d163456e1d21 admin admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:38:16.149013 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd37f7a4-2f5a-4263-85c4-9ad9fffa4673 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20d1327553404b80aecee87c78fcb52b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '559b3e713f5844b6ae37151d56dd419f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:38:16.149895 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd37f7a4-2f5a-4263-85c4-9ad9fffa4673 service cinder] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:38:16.150187 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd37f7a4-2f5a-4263-85c4-9ad9fffa4673 service cinder] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:38:16.150464 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd37f7a4-2f5a-4263-85c4-9ad9fffa4673 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:38:16.159845 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd37f7a4-2f5a-4263-85c4-9ad9fffa4673 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:38:16.161326 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd37f7a4-2f5a-4263-85c4-9ad9fffa4673 service cinder] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:38:16.170656 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1258/2509] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Jun 15 10:38:16 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:38:16.243335 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1f137a-1812-42a2-a651-c6d8e9362ab2 None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:38:16.252972 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1f137a-1812-42a2-a651-c6d8e9362ab2 None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:38:16.256593 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b92a4e-2318-4701-ba19-9e53f7618647 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20d1327553404b80aecee87c78fcb52b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '559b3e713f5844b6ae37151d56dd419f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:38:16.257233 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b92a4e-2318-4701-ba19-9e53f7618647 service cinder] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:38:16.257517 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b92a4e-2318-4701-ba19-9e53f7618647 service cinder] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:38:16.257797 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b92a4e-2318-4701-ba19-9e53f7618647 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:38:16.314005 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b92a4e-2318-4701-ba19-9e53f7618647 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:38:16.315466 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b92a4e-2318-4701-ba19-9e53f7618647 service cinder] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:38:16.324303 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1252/2510] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Jun 15 10:38:16 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:38:25.781558 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd37f7a4-2f5a-4263-85c4-9ad9fffa4673 service cinder] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:38:25.783746 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-dd37f7a4-2f5a-4263-85c4-9ad9fffa4673 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:38:25.852972 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd37f7a4-2f5a-4263-85c4-9ad9fffa4673 service cinder] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:38:25.856606 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673e6f28-69f8-4756-9986-32714e4b00a2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0cb44a17d205431793e9a8791e477a32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '559b3e713f5844b6ae37151d56dd419f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:38:25.857261 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673e6f28-69f8-4756-9986-32714e4b00a2 service neutron] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:38:25.857552 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673e6f28-69f8-4756-9986-32714e4b00a2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:38:25.857836 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673e6f28-69f8-4756-9986-32714e4b00a2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:38:25.859649 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-673e6f28-69f8-4756-9986-32714e4b00a2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:38:25.929189 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-673e6f28-69f8-4756-9986-32714e4b00a2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:38:25.930801 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-673e6f28-69f8-4756-9986-32714e4b00a2 service neutron] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:38:25.959459 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1259/2511] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Jun 15 10:38:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 178 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:38:26.320265 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=80 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:38:26.750221 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b92a4e-2318-4701-ba19-9e53f7618647 service cinder] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:38:26.762213 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b92a4e-2318-4701-ba19-9e53f7618647 service cinder] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:38:26.767359 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e81b43-cbfc-4815-8590-6e52cca808db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:38:26.768260 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e81b43-cbfc-4815-8590-6e52cca808db admin admin] REQUEST_METHOD: `DELETE` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:38:26.768610 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e81b43-cbfc-4815-8590-6e52cca808db admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:38:26.768813 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e81b43-cbfc-4815-8590-6e52cca808db admin admin] PATH_INFO: `/v3/users/d96182369bf7483a855a6daa2dde3cba` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:38:26.775479 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e81b43-cbfc-4815-8590-6e52cca808db admin admin] RBAC: Authorizing `identity:delete_user(user_id=d96182369bf7483a855a6daa2dde3cba)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:38:26.778686 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e81b43-cbfc-4815-8590-6e52cca808db admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:38:26.839761 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-76e81b43-cbfc-4815-8590-6e52cca808db admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd96182369bf7483a855a6daa2dde3cba'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:38:26.847541 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-76e81b43-cbfc-4815-8590-6e52cca808db admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd96182369bf7483a855a6daa2dde3cba'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:38:26.848363 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-76e81b43-cbfc-4815-8590-6e52cca808db admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd96182369bf7483a855a6daa2dde3cba'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:38:26.849063 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-76e81b43-cbfc-4815-8590-6e52cca808db admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd96182369bf7483a855a6daa2dde3cba'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:38:26.856643 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-76e81b43-cbfc-4815-8590-6e52cca808db admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd96182369bf7483a855a6daa2dde3cba'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:38:26.878298 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1253/2512] 10.222.0.49 () {58 vars in 1223 bytes} [Wed Jun 15 10:38:26 2022] DELETE /identity/v3/users/d96182369bf7483a855a6daa2dde3cba => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:38:26.882758 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d35b48-e46f-4848-accf-f34c6e385c1e None None] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:38:26.883441 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d35b48-e46f-4848-accf-f34c6e385c1e None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:38:26.883441 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d35b48-e46f-4848-accf-f34c6e385c1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:38:26.905607 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.auth.core [None req-29d35b48-e46f-4848-accf-f34c6e385c1e None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:38:26.906839 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-29d35b48-e46f-4848-accf-f34c6e385c1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:38:26.971387 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:38:27.004957 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1260/2513] 10.222.0.49 () {58 vars in 957 bytes} [Wed Jun 15 10:38:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:38:27.013030 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e81b43-cbfc-4815-8590-6e52cca808db admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:38:27.014839 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-76e81b43-cbfc-4815-8590-6e52cca808db admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:38:27.122489 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e81b43-cbfc-4815-8590-6e52cca808db admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:38:27.129215 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9231b76a-94aa-4e58-b350-aafa93219e68 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0cb44a17d205431793e9a8791e477a32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '559b3e713f5844b6ae37151d56dd419f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:38:27.129896 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9231b76a-94aa-4e58-b350-aafa93219e68 service neutron] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:38:27.130207 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9231b76a-94aa-4e58-b350-aafa93219e68 service neutron] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:38:27.130459 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9231b76a-94aa-4e58-b350-aafa93219e68 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:38:27.154212 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9231b76a-94aa-4e58-b350-aafa93219e68 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:38:27.157093 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9231b76a-94aa-4e58-b350-aafa93219e68 service neutron] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:38:27.168585 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1254/2514] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Jun 15 10:38:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:38:27.551808 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d35b48-e46f-4848-accf-f34c6e385c1e None None] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:38:27.553849 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-29d35b48-e46f-4848-accf-f34c6e385c1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:38:27.570418 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d35b48-e46f-4848-accf-f34c6e385c1e None None] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:38:27.581263 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7f8550-26c6-475b-8831-6b2775f06b97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:38:27.582216 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7f8550-26c6-475b-8831-6b2775f06b97 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:38:27.582560 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7f8550-26c6-475b-8831-6b2775f06b97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:38:27.582875 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7f8550-26c6-475b-8831-6b2775f06b97 admin admin] PATH_INFO: `/v3/projects/17baaaf947ab4425932593910cad8472` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:38:27.587558 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7f8550-26c6-475b-8831-6b2775f06b97 admin admin] RBAC: Authorizing `identity:delete_project(project_id=17baaaf947ab4425932593910cad8472)` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:38:27.589965 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7f8550-26c6-475b-8831-6b2775f06b97 admin admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:38:27.649874 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-ee7f8550-26c6-475b-8831-6b2775f06b97 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '17baaaf947ab4425932593910cad8472', 'request_id': 'req-ee7f8550-26c6-475b-8831-6b2775f06b97'} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:38:27.664089 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-ee7f8550-26c6-475b-8831-6b2775f06b97 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '17baaaf947ab4425932593910cad8472', 'request_id': 'req-ee7f8550-26c6-475b-8831-6b2775f06b97'} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:38:27.672656 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-ee7f8550-26c6-475b-8831-6b2775f06b97 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '17baaaf947ab4425932593910cad8472', 'request_id': 'req-ee7f8550-26c6-475b-8831-6b2775f06b97'} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:38:27.686993 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-ee7f8550-26c6-475b-8831-6b2775f06b97 admin admin] The token cache is being invalidate because project 17baaaf947ab4425932593910cad8472 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80119) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 10:38:27.687430 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-ee7f8550-26c6-475b-8831-6b2775f06b97 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:38:27.689260 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-ee7f8550-26c6-475b-8831-6b2775f06b97 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:38:27.690923 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1261/2515] 10.222.0.49 () {58 vars in 1232 bytes} [Wed Jun 15 10:38:27 2022] DELETE /identity/v3/projects/17baaaf947ab4425932593910cad8472 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:38:27.694679 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9231b76a-94aa-4e58-b350-aafa93219e68 service neutron] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:38:27.707544 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9231b76a-94aa-4e58-b350-aafa93219e68 service neutron] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:38:27.786465 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d368e021-c7db-4596-8d42-0fda4b9b1f38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:38:27.787394 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d368e021-c7db-4596-8d42-0fda4b9b1f38 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:38:27.787720 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d368e021-c7db-4596-8d42-0fda4b9b1f38 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:38:27.788296 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d368e021-c7db-4596-8d42-0fda4b9b1f38 admin admin] PATH_INFO: `/v3/users/a5afa4f0fc7f42cc8df65e637dd0e077` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:38:27.791912 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d368e021-c7db-4596-8d42-0fda4b9b1f38 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a5afa4f0fc7f42cc8df65e637dd0e077)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:38:27.794438 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d368e021-c7db-4596-8d42-0fda4b9b1f38 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:38:27.815217 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:38:27.854654 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-d368e021-c7db-4596-8d42-0fda4b9b1f38 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a5afa4f0fc7f42cc8df65e637dd0e077'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:38:27.866391 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-d368e021-c7db-4596-8d42-0fda4b9b1f38 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a5afa4f0fc7f42cc8df65e637dd0e077'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:38:27.867436 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-d368e021-c7db-4596-8d42-0fda4b9b1f38 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a5afa4f0fc7f42cc8df65e637dd0e077'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:38:27.868286 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-d368e021-c7db-4596-8d42-0fda4b9b1f38 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a5afa4f0fc7f42cc8df65e637dd0e077'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:38:27.881201 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-d368e021-c7db-4596-8d42-0fda4b9b1f38 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a5afa4f0fc7f42cc8df65e637dd0e077'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:38:27.905738 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1255/2516] 10.222.0.49 () {58 vars in 1223 bytes} [Wed Jun 15 10:38:27 2022] DELETE /identity/v3/users/a5afa4f0fc7f42cc8df65e637dd0e077 => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:38:28.215537 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7f8550-26c6-475b-8831-6b2775f06b97 admin admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:38:28.217613 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-ee7f8550-26c6-475b-8831-6b2775f06b97 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:38:28.237200 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7f8550-26c6-475b-8831-6b2775f06b97 admin admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:38:28.305487 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6028f8e3-db1a-413e-9d5e-9f21fd6db886 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:38:28.306463 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6028f8e3-db1a-413e-9d5e-9f21fd6db886 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:38:28.306817 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6028f8e3-db1a-413e-9d5e-9f21fd6db886 admin admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:38:28.307199 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6028f8e3-db1a-413e-9d5e-9f21fd6db886 admin admin] PATH_INFO: `/v3/projects/3a63f625c6274fce8a275e3bad15ce78` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:38:28.309020 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6028f8e3-db1a-413e-9d5e-9f21fd6db886 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3a63f625c6274fce8a275e3bad15ce78)` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:38:28.322402 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6028f8e3-db1a-413e-9d5e-9f21fd6db886 admin admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:38:28.378676 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-6028f8e3-db1a-413e-9d5e-9f21fd6db886 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3a63f625c6274fce8a275e3bad15ce78', 'request_id': 'req-6028f8e3-db1a-413e-9d5e-9f21fd6db886'} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:38:28.387329 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-6028f8e3-db1a-413e-9d5e-9f21fd6db886 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3a63f625c6274fce8a275e3bad15ce78', 'request_id': 'req-6028f8e3-db1a-413e-9d5e-9f21fd6db886'} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:38:28.395903 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-6028f8e3-db1a-413e-9d5e-9f21fd6db886 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3a63f625c6274fce8a275e3bad15ce78', 'request_id': 'req-6028f8e3-db1a-413e-9d5e-9f21fd6db886'} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:38:28.399246 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:38:28.412230 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-6028f8e3-db1a-413e-9d5e-9f21fd6db886 admin admin] The token cache is being invalidate because project 3a63f625c6274fce8a275e3bad15ce78 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80119) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 10:38:28.412536 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-6028f8e3-db1a-413e-9d5e-9f21fd6db886 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:38:28.414033 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-6028f8e3-db1a-413e-9d5e-9f21fd6db886 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:38:28.416341 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1262/2517] 10.222.0.49 () {58 vars in 1232 bytes} [Wed Jun 15 10:38:28 2022] DELETE /identity/v3/projects/3a63f625c6274fce8a275e3bad15ce78 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:38:28.420738 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8c9ca4-b93c-44ab-ac06-217458b4e0c0 None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:38:28.421059 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8c9ca4-b93c-44ab-ac06-217458b4e0c0 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:38:28.421341 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8c9ca4-b93c-44ab-ac06-217458b4e0c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:38:28.442937 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-ad8c9ca4-b93c-44ab-ac06-217458b4e0c0 None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:38:28.445128 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-ad8c9ca4-b93c-44ab-ac06-217458b4e0c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:38:28.534029 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1256/2518] 10.222.0.49 () {58 vars in 957 bytes} [Wed Jun 15 10:38:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:38:28.547122 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c686471-47b9-4521-bb55-0a3e05d69191 None None] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:38:28.547691 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c686471-47b9-4521-bb55-0a3e05d69191 None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:38:28.548195 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c686471-47b9-4521-bb55-0a3e05d69191 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:38:28.568852 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.auth.core [None req-9c686471-47b9-4521-bb55-0a3e05d69191 None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:38:28.570323 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-9c686471-47b9-4521-bb55-0a3e05d69191 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:38:28.580956 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1263/2519] 10.222.0.49 () {58 vars in 957 bytes} [Wed Jun 15 10:38:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:38:28.585559 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8c9ca4-b93c-44ab-ac06-217458b4e0c0 None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:38:28.600926 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8c9ca4-b93c-44ab-ac06-217458b4e0c0 None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:38:28.607318 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b1fcaf-a13a-4708-adb7-2815e5da9296 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:38:28.608718 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b1fcaf-a13a-4708-adb7-2815e5da9296 admin admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:38:28.609202 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b1fcaf-a13a-4708-adb7-2815e5da9296 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:38:28.609698 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b1fcaf-a13a-4708-adb7-2815e5da9296 admin admin] PATH_INFO: `/v3/domains` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:38:28.611058 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b1fcaf-a13a-4708-adb7-2815e5da9296 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:38:28.611900 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b1fcaf-a13a-4708-adb7-2815e5da9296 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:38:28.613862 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b1fcaf-a13a-4708-adb7-2815e5da9296 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:38:28.628350 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1257/2520] 10.222.0.49 () {58 vars in 1152 bytes} [Wed Jun 15 10:38:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:38:28.639873 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c686471-47b9-4521-bb55-0a3e05d69191 None None] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:38:28.651987 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c686471-47b9-4521-bb55-0a3e05d69191 None None] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:38:28.656595 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba14597e-f84e-4029-a160-c7ade5545059 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:38:28.658457 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba14597e-f84e-4029-a160-c7ade5545059 admin admin] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:38:28.658457 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba14597e-f84e-4029-a160-c7ade5545059 admin admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:38:28.658457 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba14597e-f84e-4029-a160-c7ade5545059 admin admin] PATH_INFO: `/v3/projects` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:38:28.659203 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba14597e-f84e-4029-a160-c7ade5545059 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:38:28.661264 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba14597e-f84e-4029-a160-c7ade5545059 admin admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:38:28.697415 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1264/2521] 10.222.0.49 () {60 vars in 1152 bytes} [Wed Jun 15 10:38:28 2022] POST /identity/v3/projects => generated 385 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 10:38:28.701524 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b1fcaf-a13a-4708-adb7-2815e5da9296 admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:38:28.713647 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b1fcaf-a13a-4708-adb7-2815e5da9296 admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:38:28.810279 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d5be98-b50b-4083-b9b3-41a0b2b2092c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:38:28.811080 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d5be98-b50b-4083-b9b3-41a0b2b2092c admin admin] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:38:28.811409 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d5be98-b50b-4083-b9b3-41a0b2b2092c admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:38:28.811693 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d5be98-b50b-4083-b9b3-41a0b2b2092c admin admin] PATH_INFO: `/v3/users` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:38:28.812412 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d5be98-b50b-4083-b9b3-41a0b2b2092c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:38:28.814871 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d5be98-b50b-4083-b9b3-41a0b2b2092c admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:38:28.835945 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1258/2522] 10.222.0.49 () {60 vars in 1143 bytes} [Wed Jun 15 10:38:28 2022] POST /identity/v3/users => generated 293 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 10:38:28.839724 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba14597e-f84e-4029-a160-c7ade5545059 admin admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:38:28.850242 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba14597e-f84e-4029-a160-c7ade5545059 admin admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:38:28.854916 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e17593-645b-46e3-a5c3-8302e30879ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:38:28.855613 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e17593-645b-46e3-a5c3-8302e30879ce admin admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:38:28.855903 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e17593-645b-46e3-a5c3-8302e30879ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:38:28.856181 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e17593-645b-46e3-a5c3-8302e30879ce admin admin] PATH_INFO: `/v3/roles` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:38:28.856918 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e17593-645b-46e3-a5c3-8302e30879ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:38:28.858524 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e17593-645b-46e3-a5c3-8302e30879ce admin admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:38:28.869550 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1265/2523] 10.222.0.49 () {58 vars in 1121 bytes} [Wed Jun 15 10:38:28 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 10:38:28.873601 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d5be98-b50b-4083-b9b3-41a0b2b2092c admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:38:28.884466 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d5be98-b50b-4083-b9b3-41a0b2b2092c admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:38:28.889402 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16af77e-c156-4f20-b6a2-2716c816b615 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:38:28.889835 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16af77e-c156-4f20-b6a2-2716c816b615 admin admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:38:28.890115 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16af77e-c156-4f20-b6a2-2716c816b615 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:38:28.890469 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16af77e-c156-4f20-b6a2-2716c816b615 admin admin] PATH_INFO: `/v3/roles` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:38:28.891286 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a16af77e-c156-4f20-b6a2-2716c816b615 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:38:28.893436 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a16af77e-c156-4f20-b6a2-2716c816b615 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:38:28.901733 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:DELETE=9,keystone:SELECT=87,keystone:INSERT=4 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:38:28.909469 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1259/2524] 10.222.0.49 () {58 vars in 1121 bytes} [Wed Jun 15 10:38:28 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 10:38:28.913644 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e17593-645b-46e3-a5c3-8302e30879ce admin admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:38:28.923964 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e17593-645b-46e3-a5c3-8302e30879ce admin admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:38:28.928720 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b208e872-fbdc-4f5e-934d-a0b68bd3c7d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:38:28.929717 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b208e872-fbdc-4f5e-934d-a0b68bd3c7d5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:38:28.929880 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b208e872-fbdc-4f5e-934d-a0b68bd3c7d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:38:28.930167 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b208e872-fbdc-4f5e-934d-a0b68bd3c7d5 admin admin] PATH_INFO: `/v3/projects/7efe1e4c86ad424aaad415f1663d4619/users/873b3e33e2b944a7a6daec8e3f55c5d0/roles/6876e57cc51c448097ab109d35b2a3da` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:38:28.946431 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b208e872-fbdc-4f5e-934d-a0b68bd3c7d5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7efe1e4c86ad424aaad415f1663d4619, user_id=873b3e33e2b944a7a6daec8e3f55c5d0, role_id=6876e57cc51c448097ab109d35b2a3da)` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:38:28.949316 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b208e872-fbdc-4f5e-934d-a0b68bd3c7d5 admin admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:38:28.963744 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1266/2525] 10.222.0.49 () {60 vars in 1482 bytes} [Wed Jun 15 10:38:28 2022] PUT /identity/v3/projects/7efe1e4c86ad424aaad415f1663d4619/users/873b3e33e2b944a7a6daec8e3f55c5d0/roles/6876e57cc51c448097ab109d35b2a3da => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:38:28.975226 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284d5f37-495d-4d4c-aece-7499fb781cb7 None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:38:28.975518 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284d5f37-495d-4d4c-aece-7499fb781cb7 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:38:28.975798 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284d5f37-495d-4d4c-aece-7499fb781cb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:38:29.003241 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-284d5f37-495d-4d4c-aece-7499fb781cb7 None None] MFA Rules not processed for user `873b3e33e2b944a7a6daec8e3f55c5d0`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:38:29.004462 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-284d5f37-495d-4d4c-aece-7499fb781cb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:38:29.084254 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1260/2526] 10.222.0.49 () {58 vars in 957 bytes} [Wed Jun 15 10:38:28 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:38:29.088894 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b208e872-fbdc-4f5e-934d-a0b68bd3c7d5 admin admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:38:29.168280 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b208e872-fbdc-4f5e-934d-a0b68bd3c7d5 admin admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:38:29.173241 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75f77bc-6cd2-48d0-88ee-6828cd16f95c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:38:29.173913 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75f77bc-6cd2-48d0-88ee-6828cd16f95c admin admin] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:38:29.174202 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75f77bc-6cd2-48d0-88ee-6828cd16f95c admin admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:38:29.174479 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75f77bc-6cd2-48d0-88ee-6828cd16f95c admin admin] PATH_INFO: `/v3/projects` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:38:29.175201 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75f77bc-6cd2-48d0-88ee-6828cd16f95c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:38:29.177131 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75f77bc-6cd2-48d0-88ee-6828cd16f95c admin admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:38:29.206003 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1267/2527] 10.222.0.49 () {60 vars in 1152 bytes} [Wed Jun 15 10:38:29 2022] POST /identity/v3/projects => generated 385 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 10:38:29.210100 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284d5f37-495d-4d4c-aece-7499fb781cb7 None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:38:29.223116 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284d5f37-495d-4d4c-aece-7499fb781cb7 None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:38:29.292819 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c27f945-dc78-4b49-81a9-94a5b464f8b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:38:29.293505 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c27f945-dc78-4b49-81a9-94a5b464f8b5 admin admin] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:38:29.293819 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c27f945-dc78-4b49-81a9-94a5b464f8b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:38:29.294066 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c27f945-dc78-4b49-81a9-94a5b464f8b5 admin admin] PATH_INFO: `/v3/users` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:38:29.294753 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c27f945-dc78-4b49-81a9-94a5b464f8b5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:38:29.296708 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c27f945-dc78-4b49-81a9-94a5b464f8b5 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:38:29.317764 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1261/2528] 10.222.0.49 () {60 vars in 1143 bytes} [Wed Jun 15 10:38:29 2022] POST /identity/v3/users => generated 299 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 10:38:29.322365 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75f77bc-6cd2-48d0-88ee-6828cd16f95c admin admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:38:29.339224 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75f77bc-6cd2-48d0-88ee-6828cd16f95c admin admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:38:29.345255 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9334f04e-4485-4b9f-9463-77a3f07206b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:38:29.346252 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9334f04e-4485-4b9f-9463-77a3f07206b3 admin admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:38:29.346613 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9334f04e-4485-4b9f-9463-77a3f07206b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:38:29.346989 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9334f04e-4485-4b9f-9463-77a3f07206b3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:38:29.348060 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9334f04e-4485-4b9f-9463-77a3f07206b3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:38:29.350169 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9334f04e-4485-4b9f-9463-77a3f07206b3 admin admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:38:29.366661 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1268/2529] 10.222.0.49 () {58 vars in 1121 bytes} [Wed Jun 15 10:38:29 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 15 10:38:29.367959 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c27f945-dc78-4b49-81a9-94a5b464f8b5 admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:38:29.380394 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c27f945-dc78-4b49-81a9-94a5b464f8b5 admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:38:29.390078 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7964a4-3567-4323-bb20-f180fc69f8cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:38:29.391274 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7964a4-3567-4323-bb20-f180fc69f8cd admin admin] REQUEST_METHOD: `PUT` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:38:29.392195 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7964a4-3567-4323-bb20-f180fc69f8cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:38:29.392567 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7964a4-3567-4323-bb20-f180fc69f8cd admin admin] PATH_INFO: `/v3/projects/090490cf141f45d9a00afe05675fed43/users/af006d3f6cbd4746b597ea93ab0d21b1/roles/8051174e7a7947d5b991ebd793c2e543` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:38:29.412757 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7964a4-3567-4323-bb20-f180fc69f8cd admin admin] RBAC: Authorizing `identity:create_grant(project_id=090490cf141f45d9a00afe05675fed43, user_id=af006d3f6cbd4746b597ea93ab0d21b1, role_id=8051174e7a7947d5b991ebd793c2e543)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:38:29.416377 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7964a4-3567-4323-bb20-f180fc69f8cd admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:38:29.435193 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1262/2530] 10.222.0.49 () {60 vars in 1482 bytes} [Wed Jun 15 10:38:29 2022] PUT /identity/v3/projects/090490cf141f45d9a00afe05675fed43/users/af006d3f6cbd4746b597ea93ab0d21b1/roles/8051174e7a7947d5b991ebd793c2e543 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:38:29.439768 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9334f04e-4485-4b9f-9463-77a3f07206b3 admin admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:38:29.450358 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9334f04e-4485-4b9f-9463-77a3f07206b3 admin admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:38:29.524031 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e51f2af-2f37-4706-92f5-e3d04a9746d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:38:29.524979 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e51f2af-2f37-4706-92f5-e3d04a9746d1 admin admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:38:29.525266 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e51f2af-2f37-4706-92f5-e3d04a9746d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:38:29.525546 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e51f2af-2f37-4706-92f5-e3d04a9746d1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:38:29.526274 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e51f2af-2f37-4706-92f5-e3d04a9746d1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:38:29.527935 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e51f2af-2f37-4706-92f5-e3d04a9746d1 admin admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:38:29.539331 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1269/2531] 10.222.0.49 () {58 vars in 1121 bytes} [Wed Jun 15 10:38:29 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 15 10:38:29.543371 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7964a4-3567-4323-bb20-f180fc69f8cd admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:38:29.555504 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7964a4-3567-4323-bb20-f180fc69f8cd admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:38:29.561446 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165e9de8-8cd0-4840-ae6a-577220cb73ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:38:29.562814 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165e9de8-8cd0-4840-ae6a-577220cb73ef admin admin] REQUEST_METHOD: `PUT` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:38:29.562814 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165e9de8-8cd0-4840-ae6a-577220cb73ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:38:29.563065 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165e9de8-8cd0-4840-ae6a-577220cb73ef admin admin] PATH_INFO: `/v3/projects/090490cf141f45d9a00afe05675fed43/users/af006d3f6cbd4746b597ea93ab0d21b1/roles/8051174e7a7947d5b991ebd793c2e543` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:38:29.565148 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-165e9de8-8cd0-4840-ae6a-577220cb73ef admin admin] RBAC: Authorizing `identity:create_grant(project_id=090490cf141f45d9a00afe05675fed43, user_id=af006d3f6cbd4746b597ea93ab0d21b1, role_id=8051174e7a7947d5b991ebd793c2e543)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:38:29.567846 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-165e9de8-8cd0-4840-ae6a-577220cb73ef admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:38:29.581153 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1263/2532] 10.222.0.49 () {60 vars in 1482 bytes} [Wed Jun 15 10:38:29 2022] PUT /identity/v3/projects/090490cf141f45d9a00afe05675fed43/users/af006d3f6cbd4746b597ea93ab0d21b1/roles/8051174e7a7947d5b991ebd793c2e543 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 10:38:29.596037 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491cd250-0ff2-425f-b217-d8d0617ff7d5 None None] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:38:29.596037 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491cd250-0ff2-425f-b217-d8d0617ff7d5 None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:38:29.596037 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491cd250-0ff2-425f-b217-d8d0617ff7d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:38:29.600566 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:38:29.625335 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.auth.core [None req-491cd250-0ff2-425f-b217-d8d0617ff7d5 None None] MFA Rules not processed for user `af006d3f6cbd4746b597ea93ab0d21b1`. Rule list: `[]` (Enabled: `True`). {{(pid=80119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:38:29.626667 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-491cd250-0ff2-425f-b217-d8d0617ff7d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:38:29.698775 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1270/2533] 10.222.0.49 () {58 vars in 957 bytes} [Wed Jun 15 10:38:29 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:38:29.709833 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165e9de8-8cd0-4840-ae6a-577220cb73ef admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:38:29.711644 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-165e9de8-8cd0-4840-ae6a-577220cb73ef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:38:29.760723 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:38:29.774139 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165e9de8-8cd0-4840-ae6a-577220cb73ef admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:38:29.778344 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37718b6-e1e3-4fa0-9805-2f3cd26f077b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20d1327553404b80aecee87c78fcb52b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '559b3e713f5844b6ae37151d56dd419f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:38:29.779148 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37718b6-e1e3-4fa0-9805-2f3cd26f077b service cinder] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:38:29.779541 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37718b6-e1e3-4fa0-9805-2f3cd26f077b service cinder] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:38:29.779922 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37718b6-e1e3-4fa0-9805-2f3cd26f077b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:38:29.901472 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e37718b6-e1e3-4fa0-9805-2f3cd26f077b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:38:29.902786 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e37718b6-e1e3-4fa0-9805-2f3cd26f077b service cinder] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:38:29.914302 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1264/2534] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Jun 15 10:38:29 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 205 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:38:30.022268 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491cd250-0ff2-425f-b217-d8d0617ff7d5 None None] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:38:30.025242 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-491cd250-0ff2-425f-b217-d8d0617ff7d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:38:30.122096 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491cd250-0ff2-425f-b217-d8d0617ff7d5 None None] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:38:30.126831 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded76a01-a98a-47c3-8430-d25ef23aaf02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:38:30.127762 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded76a01-a98a-47c3-8430-d25ef23aaf02 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:38:30.128046 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded76a01-a98a-47c3-8430-d25ef23aaf02 admin admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:38:30.128320 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded76a01-a98a-47c3-8430-d25ef23aaf02 admin admin] PATH_INFO: `/v3/users/9ca22364db8147f2a14f3a0c77f545de` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:38:30.129517 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded76a01-a98a-47c3-8430-d25ef23aaf02 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9ca22364db8147f2a14f3a0c77f545de)` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:38:30.131468 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded76a01-a98a-47c3-8430-d25ef23aaf02 admin admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:38:30.181794 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-ded76a01-a98a-47c3-8430-d25ef23aaf02 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9ca22364db8147f2a14f3a0c77f545de'} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:38:30.192429 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-ded76a01-a98a-47c3-8430-d25ef23aaf02 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9ca22364db8147f2a14f3a0c77f545de'} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:38:30.193535 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-ded76a01-a98a-47c3-8430-d25ef23aaf02 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9ca22364db8147f2a14f3a0c77f545de'} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:38:30.194376 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-ded76a01-a98a-47c3-8430-d25ef23aaf02 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9ca22364db8147f2a14f3a0c77f545de'} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:38:30.203736 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-ded76a01-a98a-47c3-8430-d25ef23aaf02 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9ca22364db8147f2a14f3a0c77f545de'} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:38:30.222764 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1271/2535] 10.222.0.49 () {58 vars in 1223 bytes} [Wed Jun 15 10:38:30 2022] DELETE /identity/v3/users/9ca22364db8147f2a14f3a0c77f545de => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:38:30.418409 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37718b6-e1e3-4fa0-9805-2f3cd26f077b service cinder] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:38:30.431552 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37718b6-e1e3-4fa0-9805-2f3cd26f077b service cinder] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:38:30.499350 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc4adb2-03d3-4866-9a21-c6fd74caca78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:38:30.500229 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc4adb2-03d3-4866-9a21-c6fd74caca78 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:38:30.500620 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc4adb2-03d3-4866-9a21-c6fd74caca78 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:38:30.501414 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc4adb2-03d3-4866-9a21-c6fd74caca78 admin admin] PATH_INFO: `/v3/projects/d5320ba7f73d40cb9a6d16225d6660c7` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:38:30.502647 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc4adb2-03d3-4866-9a21-c6fd74caca78 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d5320ba7f73d40cb9a6d16225d6660c7)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:38:30.504583 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc4adb2-03d3-4866-9a21-c6fd74caca78 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:38:30.551749 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-ccc4adb2-03d3-4866-9a21-c6fd74caca78 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd5320ba7f73d40cb9a6d16225d6660c7', 'request_id': 'req-ccc4adb2-03d3-4866-9a21-c6fd74caca78'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:38:30.558938 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-ccc4adb2-03d3-4866-9a21-c6fd74caca78 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd5320ba7f73d40cb9a6d16225d6660c7', 'request_id': 'req-ccc4adb2-03d3-4866-9a21-c6fd74caca78'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:38:30.564986 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-ccc4adb2-03d3-4866-9a21-c6fd74caca78 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd5320ba7f73d40cb9a6d16225d6660c7', 'request_id': 'req-ccc4adb2-03d3-4866-9a21-c6fd74caca78'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:38:30.575712 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-ccc4adb2-03d3-4866-9a21-c6fd74caca78 admin admin] The token cache is being invalidate because project d5320ba7f73d40cb9a6d16225d6660c7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80120) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 10:38:30.576004 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-ccc4adb2-03d3-4866-9a21-c6fd74caca78 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:38:30.576634 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-ccc4adb2-03d3-4866-9a21-c6fd74caca78 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:38:30.578192 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1265/2536] 10.222.0.49 () {58 vars in 1232 bytes} [Wed Jun 15 10:38:30 2022] DELETE /identity/v3/projects/d5320ba7f73d40cb9a6d16225d6660c7 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:38:30.583321 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e15cca8-24fb-4f5d-a1b7-2e7b5a9c6313 None None] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:38:30.583781 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e15cca8-24fb-4f5d-a1b7-2e7b5a9c6313 None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:38:30.583920 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e15cca8-24fb-4f5d-a1b7-2e7b5a9c6313 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:38:30.596117 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:38:30.605209 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.auth.core [None req-7e15cca8-24fb-4f5d-a1b7-2e7b5a9c6313 None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:38:30.606955 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-7e15cca8-24fb-4f5d-a1b7-2e7b5a9c6313 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:38:30.668367 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1272/2537] 10.222.0.49 () {58 vars in 957 bytes} [Wed Jun 15 10:38:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:38:30.676754 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c3567a-e429-4888-98a5-24c85cba22ba None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:38:30.677127 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c3567a-e429-4888-98a5-24c85cba22ba None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:38:30.677477 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c3567a-e429-4888-98a5-24c85cba22ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:38:30.696223 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-15c3567a-e429-4888-98a5-24c85cba22ba None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:38:30.697448 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-15c3567a-e429-4888-98a5-24c85cba22ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:38:30.705428 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1266/2538] 10.222.0.49 () {58 vars in 957 bytes} [Wed Jun 15 10:38:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:38:30.709631 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e15cca8-24fb-4f5d-a1b7-2e7b5a9c6313 None None] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:38:30.719264 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e15cca8-24fb-4f5d-a1b7-2e7b5a9c6313 None None] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:38:30.723766 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d914efc-68fc-445b-a7bd-0bfa8230968b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:38:30.724416 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d914efc-68fc-445b-a7bd-0bfa8230968b admin admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:38:30.724699 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d914efc-68fc-445b-a7bd-0bfa8230968b admin admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:38:30.724970 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d914efc-68fc-445b-a7bd-0bfa8230968b admin admin] PATH_INFO: `/v3/domains` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:38:30.725678 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d914efc-68fc-445b-a7bd-0bfa8230968b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:38:30.725980 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d914efc-68fc-445b-a7bd-0bfa8230968b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:38:30.727583 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d914efc-68fc-445b-a7bd-0bfa8230968b admin admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:38:30.739359 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1273/2539] 10.222.0.49 () {58 vars in 1152 bytes} [Wed Jun 15 10:38:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:38:30.743594 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c3567a-e429-4888-98a5-24c85cba22ba None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:38:30.754587 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c3567a-e429-4888-98a5-24c85cba22ba None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:38:30.759408 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c08096-296b-42a4-9e9c-6988f9d8f55d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:38:30.760103 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c08096-296b-42a4-9e9c-6988f9d8f55d admin admin] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:38:30.760394 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c08096-296b-42a4-9e9c-6988f9d8f55d admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:38:30.760673 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c08096-296b-42a4-9e9c-6988f9d8f55d admin admin] PATH_INFO: `/v3/projects` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:38:30.761362 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c08096-296b-42a4-9e9c-6988f9d8f55d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:38:30.763294 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c08096-296b-42a4-9e9c-6988f9d8f55d admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:38:30.783036 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:38:30.792514 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1267/2540] 10.222.0.49 () {60 vars in 1152 bytes} [Wed Jun 15 10:38:30 2022] POST /identity/v3/projects => generated 373 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 10:38:30.796219 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d914efc-68fc-445b-a7bd-0bfa8230968b admin admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:38:30.807963 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d914efc-68fc-445b-a7bd-0bfa8230968b admin admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:38:30.863770 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f234fbd-2a01-4e4c-b26e-cd5e3aa0b773 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:38:30.864441 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f234fbd-2a01-4e4c-b26e-cd5e3aa0b773 admin admin] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:38:30.864730 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f234fbd-2a01-4e4c-b26e-cd5e3aa0b773 admin admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:38:30.865011 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f234fbd-2a01-4e4c-b26e-cd5e3aa0b773 admin admin] PATH_INFO: `/v3/users` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:38:30.865701 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f234fbd-2a01-4e4c-b26e-cd5e3aa0b773 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:38:30.867660 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f234fbd-2a01-4e4c-b26e-cd5e3aa0b773 admin admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:38:30.887681 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1274/2541] 10.222.0.49 () {60 vars in 1143 bytes} [Wed Jun 15 10:38:30 2022] POST /identity/v3/users => generated 287 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 10:38:30.891711 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c08096-296b-42a4-9e9c-6988f9d8f55d admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:38:30.902324 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c08096-296b-42a4-9e9c-6988f9d8f55d admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:38:30.906945 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4014a2-490b-4caa-b624-f95c4d93485a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:38:30.908242 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4014a2-490b-4caa-b624-f95c4d93485a admin admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:38:30.908558 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4014a2-490b-4caa-b624-f95c4d93485a admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:38:30.908845 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4014a2-490b-4caa-b624-f95c4d93485a admin admin] PATH_INFO: `/v3/roles` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:38:30.909577 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d4014a2-490b-4caa-b624-f95c4d93485a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:38:30.911197 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d4014a2-490b-4caa-b624-f95c4d93485a admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:38:30.922791 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1268/2542] 10.222.0.49 () {58 vars in 1121 bytes} [Wed Jun 15 10:38:30 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 10:38:30.927160 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f234fbd-2a01-4e4c-b26e-cd5e3aa0b773 admin admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:38:30.936966 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f234fbd-2a01-4e4c-b26e-cd5e3aa0b773 admin admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:38:30.941606 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f417f3c0-724d-4052-ac66-278370d3c1d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:38:30.942258 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f417f3c0-724d-4052-ac66-278370d3c1d0 admin admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:38:30.942540 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f417f3c0-724d-4052-ac66-278370d3c1d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:38:30.942816 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f417f3c0-724d-4052-ac66-278370d3c1d0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:38:30.943561 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f417f3c0-724d-4052-ac66-278370d3c1d0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:38:30.945143 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f417f3c0-724d-4052-ac66-278370d3c1d0 admin admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:38:30.954907 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1275/2543] 10.222.0.49 () {58 vars in 1121 bytes} [Wed Jun 15 10:38:30 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 10:38:30.958838 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4014a2-490b-4caa-b624-f95c4d93485a admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:38:30.968761 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4014a2-490b-4caa-b624-f95c4d93485a admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:38:30.974097 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b61b4c8-ab24-4881-836a-d2219d88a2f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:38:30.974908 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b61b4c8-ab24-4881-836a-d2219d88a2f9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:38:30.975232 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b61b4c8-ab24-4881-836a-d2219d88a2f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:38:30.975514 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b61b4c8-ab24-4881-836a-d2219d88a2f9 admin admin] PATH_INFO: `/v3/projects/8317ecae5bc54a03ac059871ade2e3b5/users/cb88d15829b941db99ed9071040eec1a/roles/6876e57cc51c448097ab109d35b2a3da` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:38:30.990329 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b61b4c8-ab24-4881-836a-d2219d88a2f9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8317ecae5bc54a03ac059871ade2e3b5, user_id=cb88d15829b941db99ed9071040eec1a, role_id=6876e57cc51c448097ab109d35b2a3da)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:38:30.993019 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b61b4c8-ab24-4881-836a-d2219d88a2f9 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:38:31.005516 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1269/2544] 10.222.0.49 () {60 vars in 1482 bytes} [Wed Jun 15 10:38:30 2022] PUT /identity/v3/projects/8317ecae5bc54a03ac059871ade2e3b5/users/cb88d15829b941db99ed9071040eec1a/roles/6876e57cc51c448097ab109d35b2a3da => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:38:31.016197 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc37f6b0-0928-4a03-b16b-b3fb50ee1268 None None] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:38:31.016488 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc37f6b0-0928-4a03-b16b-b3fb50ee1268 None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:38:31.016810 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc37f6b0-0928-4a03-b16b-b3fb50ee1268 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:38:31.043148 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.auth.core [None req-fc37f6b0-0928-4a03-b16b-b3fb50ee1268 None None] MFA Rules not processed for user `cb88d15829b941db99ed9071040eec1a`. Rule list: `[]` (Enabled: `True`). {{(pid=80119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:38:31.044366 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-fc37f6b0-0928-4a03-b16b-b3fb50ee1268 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:38:31.106130 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1276/2545] 10.222.0.49 () {58 vars in 957 bytes} [Wed Jun 15 10:38:31 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:38:31.118125 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b61b4c8-ab24-4881-836a-d2219d88a2f9 admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:38:31.119804 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-1b61b4c8-ab24-4881-836a-d2219d88a2f9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:38:31.205514 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b61b4c8-ab24-4881-836a-d2219d88a2f9 admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:38:31.211435 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165c4375-282a-4323-af81-2a8762b23109 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ac54528370d4747b6e8777fba9d0c82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '559b3e713f5844b6ae37151d56dd419f', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:38:31.212106 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165c4375-282a-4323-af81-2a8762b23109 service nova] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:38:31.212451 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165c4375-282a-4323-af81-2a8762b23109 service nova] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:38:31.212805 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165c4375-282a-4323-af81-2a8762b23109 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:38:31.222660 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-165c4375-282a-4323-af81-2a8762b23109 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:38:31.224445 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-165c4375-282a-4323-af81-2a8762b23109 service nova] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:38:31.232037 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc37f6b0-0928-4a03-b16b-b3fb50ee1268 None None] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:38:31.233146 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-fc37f6b0-0928-4a03-b16b-b3fb50ee1268 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:38:31.233959 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1270/2546] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Jun 15 10:38:31 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:38:31.239468 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:38:31.295369 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc37f6b0-0928-4a03-b16b-b3fb50ee1268 None None] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:38:31.299876 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c23b06-98fa-47ed-ab4b-090c77eef51e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0cb44a17d205431793e9a8791e477a32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '559b3e713f5844b6ae37151d56dd419f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:38:31.300540 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c23b06-98fa-47ed-ab4b-090c77eef51e service neutron] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:38:31.303404 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c23b06-98fa-47ed-ab4b-090c77eef51e service neutron] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:38:31.303404 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c23b06-98fa-47ed-ab4b-090c77eef51e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:38:31.306609 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-a4c23b06-98fa-47ed-ab4b-090c77eef51e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:38:31.393744 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c23b06-98fa-47ed-ab4b-090c77eef51e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:38:31.393744 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c23b06-98fa-47ed-ab4b-090c77eef51e service neutron] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:38:31.402751 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1277/2547] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Jun 15 10:38:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 10:38:31.475699 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165c4375-282a-4323-af81-2a8762b23109 service nova] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:38:31.486510 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165c4375-282a-4323-af81-2a8762b23109 service nova] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:38:31.492293 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a44aab-ba5a-41a6-9b5c-72eb60e6b980 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:38:31.493386 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a44aab-ba5a-41a6-9b5c-72eb60e6b980 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:38:31.493677 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a44aab-ba5a-41a6-9b5c-72eb60e6b980 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:38:31.494301 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a44aab-ba5a-41a6-9b5c-72eb60e6b980 admin admin] PATH_INFO: `/v3/users/cb88d15829b941db99ed9071040eec1a` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:38:31.495242 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a44aab-ba5a-41a6-9b5c-72eb60e6b980 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cb88d15829b941db99ed9071040eec1a)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:38:31.497849 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a44aab-ba5a-41a6-9b5c-72eb60e6b980 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:38:31.554738 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-b4a44aab-ba5a-41a6-9b5c-72eb60e6b980 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cb88d15829b941db99ed9071040eec1a'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:38:31.562251 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-b4a44aab-ba5a-41a6-9b5c-72eb60e6b980 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cb88d15829b941db99ed9071040eec1a'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:38:31.563535 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-b4a44aab-ba5a-41a6-9b5c-72eb60e6b980 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cb88d15829b941db99ed9071040eec1a'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:38:31.564165 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-b4a44aab-ba5a-41a6-9b5c-72eb60e6b980 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cb88d15829b941db99ed9071040eec1a'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:38:31.572688 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-b4a44aab-ba5a-41a6-9b5c-72eb60e6b980 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cb88d15829b941db99ed9071040eec1a'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:38:31.594957 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1271/2548] 10.222.0.49 () {58 vars in 1223 bytes} [Wed Jun 15 10:38:31 2022] DELETE /identity/v3/users/cb88d15829b941db99ed9071040eec1a => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:38:31.599730 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d017e0ed-8fba-4ba3-8f09-6949277781b5 None None] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:38:31.600179 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d017e0ed-8fba-4ba3-8f09-6949277781b5 None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:38:31.600399 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d017e0ed-8fba-4ba3-8f09-6949277781b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:38:31.618323 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.auth.core [None req-d017e0ed-8fba-4ba3-8f09-6949277781b5 None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:38:31.619997 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-d017e0ed-8fba-4ba3-8f09-6949277781b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:38:31.716211 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1278/2549] 10.222.0.49 () {58 vars in 957 bytes} [Wed Jun 15 10:38:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:38:31.728400 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a44aab-ba5a-41a6-9b5c-72eb60e6b980 admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:38:31.731709 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-b4a44aab-ba5a-41a6-9b5c-72eb60e6b980 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:38:31.772825 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:38:31.821806 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a44aab-ba5a-41a6-9b5c-72eb60e6b980 admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:38:31.826199 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a02d3ed-e4d6-4558-bfdb-8a907bcf6ebf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0cb44a17d205431793e9a8791e477a32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '559b3e713f5844b6ae37151d56dd419f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:38:31.826961 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a02d3ed-e4d6-4558-bfdb-8a907bcf6ebf service neutron] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:38:31.827361 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a02d3ed-e4d6-4558-bfdb-8a907bcf6ebf service neutron] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:38:31.827713 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a02d3ed-e4d6-4558-bfdb-8a907bcf6ebf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:38:31.838598 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a02d3ed-e4d6-4558-bfdb-8a907bcf6ebf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:38:31.840852 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a02d3ed-e4d6-4558-bfdb-8a907bcf6ebf service neutron] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:38:31.855366 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1272/2550] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Jun 15 10:38:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:38:32.165535 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d017e0ed-8fba-4ba3-8f09-6949277781b5 None None] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:38:32.167006 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-d017e0ed-8fba-4ba3-8f09-6949277781b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:38:32.182266 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d017e0ed-8fba-4ba3-8f09-6949277781b5 None None] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:38:32.189054 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1801aa-fd0d-467a-a799-a318b98eaf07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:38:32.190031 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1801aa-fd0d-467a-a799-a318b98eaf07 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:38:32.190404 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1801aa-fd0d-467a-a799-a318b98eaf07 admin admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:38:32.190729 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1801aa-fd0d-467a-a799-a318b98eaf07 admin admin] PATH_INFO: `/v3/projects/8317ecae5bc54a03ac059871ade2e3b5` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:38:32.192626 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1801aa-fd0d-467a-a799-a318b98eaf07 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8317ecae5bc54a03ac059871ade2e3b5)` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:38:32.194726 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1801aa-fd0d-467a-a799-a318b98eaf07 admin admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:38:32.262613 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:38:32.264604 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-3b1801aa-fd0d-467a-a799-a318b98eaf07 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8317ecae5bc54a03ac059871ade2e3b5', 'request_id': 'req-3b1801aa-fd0d-467a-a799-a318b98eaf07'} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:38:32.275372 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-3b1801aa-fd0d-467a-a799-a318b98eaf07 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8317ecae5bc54a03ac059871ade2e3b5', 'request_id': 'req-3b1801aa-fd0d-467a-a799-a318b98eaf07'} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:38:32.282983 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-3b1801aa-fd0d-467a-a799-a318b98eaf07 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8317ecae5bc54a03ac059871ade2e3b5', 'request_id': 'req-3b1801aa-fd0d-467a-a799-a318b98eaf07'} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:38:32.299224 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-3b1801aa-fd0d-467a-a799-a318b98eaf07 admin admin] The token cache is being invalidate because project 8317ecae5bc54a03ac059871ade2e3b5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80119) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 10:38:32.299656 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-3b1801aa-fd0d-467a-a799-a318b98eaf07 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:38:32.300405 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-3b1801aa-fd0d-467a-a799-a318b98eaf07 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:38:32.302849 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1279/2551] 10.222.0.49 () {58 vars in 1232 bytes} [Wed Jun 15 10:38:32 2022] DELETE /identity/v3/projects/8317ecae5bc54a03ac059871ade2e3b5 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:38:32.308601 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d17c914-6963-4e18-8585-7663a7322362 None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:38:32.309102 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d17c914-6963-4e18-8585-7663a7322362 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:38:32.309565 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d17c914-6963-4e18-8585-7663a7322362 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:38:32.329499 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-8d17c914-6963-4e18-8585-7663a7322362 None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:38:32.330845 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-8d17c914-6963-4e18-8585-7663a7322362 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:38:32.397958 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1273/2552] 10.222.0.49 () {58 vars in 957 bytes} [Wed Jun 15 10:38:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 10:38:32.405694 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec688fe-47c6-4f68-846b-19585ddb72d2 None None] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:38:32.405989 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec688fe-47c6-4f68-846b-19585ddb72d2 None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:38:32.406270 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec688fe-47c6-4f68-846b-19585ddb72d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:38:32.429244 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.auth.core [None req-8ec688fe-47c6-4f68-846b-19585ddb72d2 None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:38:32.430490 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-8ec688fe-47c6-4f68-846b-19585ddb72d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:38:32.442474 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1280/2553] 10.222.0.49 () {58 vars in 957 bytes} [Wed Jun 15 10:38:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:38:32.447540 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d17c914-6963-4e18-8585-7663a7322362 None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:38:32.457873 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d17c914-6963-4e18-8585-7663a7322362 None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:38:32.463855 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c440ce-bc44-4dde-9450-de4174ab7413 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:38:32.464664 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c440ce-bc44-4dde-9450-de4174ab7413 admin admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:38:32.464957 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c440ce-bc44-4dde-9450-de4174ab7413 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:38:32.465240 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c440ce-bc44-4dde-9450-de4174ab7413 admin admin] PATH_INFO: `/v3/domains` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:38:32.465996 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c440ce-bc44-4dde-9450-de4174ab7413 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:38:32.466293 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c440ce-bc44-4dde-9450-de4174ab7413 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:38:32.468118 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c440ce-bc44-4dde-9450-de4174ab7413 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:38:32.487502 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1274/2554] 10.222.0.49 () {58 vars in 1152 bytes} [Wed Jun 15 10:38:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:38:32.490269 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec688fe-47c6-4f68-846b-19585ddb72d2 None None] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:38:32.501970 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec688fe-47c6-4f68-846b-19585ddb72d2 None None] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:38:32.506835 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f19432-b9e4-4eb7-a1f8-8897ca229642 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:38:32.508546 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f19432-b9e4-4eb7-a1f8-8897ca229642 admin admin] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:38:32.508546 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f19432-b9e4-4eb7-a1f8-8897ca229642 admin admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:38:32.509617 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f19432-b9e4-4eb7-a1f8-8897ca229642 admin admin] PATH_INFO: `/v3/projects` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:38:32.510301 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f19432-b9e4-4eb7-a1f8-8897ca229642 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:38:32.512279 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f19432-b9e4-4eb7-a1f8-8897ca229642 admin admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:38:32.547139 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1281/2555] 10.222.0.49 () {60 vars in 1152 bytes} [Wed Jun 15 10:38:32 2022] POST /identity/v3/projects => generated 393 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 10:38:32.549936 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c440ce-bc44-4dde-9450-de4174ab7413 admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:38:32.561469 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c440ce-bc44-4dde-9450-de4174ab7413 admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:38:32.632910 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d995936-8050-43b7-a1ab-189a9e7d9c4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:38:32.633608 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d995936-8050-43b7-a1ab-189a9e7d9c4f admin admin] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:38:32.633963 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d995936-8050-43b7-a1ab-189a9e7d9c4f admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:38:32.634246 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d995936-8050-43b7-a1ab-189a9e7d9c4f admin admin] PATH_INFO: `/v3/users` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:38:32.634947 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d995936-8050-43b7-a1ab-189a9e7d9c4f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:38:32.636967 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d995936-8050-43b7-a1ab-189a9e7d9c4f admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:38:32.674855 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1275/2556] 10.222.0.49 () {60 vars in 1143 bytes} [Wed Jun 15 10:38:32 2022] POST /identity/v3/users => generated 297 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 10:38:32.676200 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f19432-b9e4-4eb7-a1f8-8897ca229642 admin admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:38:32.686938 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f19432-b9e4-4eb7-a1f8-8897ca229642 admin admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:38:32.692462 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16bd9cb8-30df-4a5a-9523-d1489589567e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:38:32.693081 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bd9cb8-30df-4a5a-9523-d1489589567e admin admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:38:32.693366 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bd9cb8-30df-4a5a-9523-d1489589567e admin admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:38:32.693643 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bd9cb8-30df-4a5a-9523-d1489589567e admin admin] PATH_INFO: `/v3/roles` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:38:32.694379 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16bd9cb8-30df-4a5a-9523-d1489589567e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:38:32.696041 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16bd9cb8-30df-4a5a-9523-d1489589567e admin admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:38:32.709272 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1282/2557] 10.222.0.49 () {58 vars in 1121 bytes} [Wed Jun 15 10:38:32 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 10:38:32.713585 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d995936-8050-43b7-a1ab-189a9e7d9c4f admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:38:32.724220 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d995936-8050-43b7-a1ab-189a9e7d9c4f admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:38:32.729197 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508661de-a873-4b67-be13-47e3781192fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:38:32.729843 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508661de-a873-4b67-be13-47e3781192fe admin admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:38:32.730129 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508661de-a873-4b67-be13-47e3781192fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:38:32.730407 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508661de-a873-4b67-be13-47e3781192fe admin admin] PATH_INFO: `/v3/roles` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:38:32.731183 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508661de-a873-4b67-be13-47e3781192fe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:38:32.732797 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508661de-a873-4b67-be13-47e3781192fe admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:38:32.746912 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1276/2558] 10.222.0.49 () {58 vars in 1121 bytes} [Wed Jun 15 10:38:32 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 10:38:32.747861 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16bd9cb8-30df-4a5a-9523-d1489589567e admin admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:38:32.758463 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16bd9cb8-30df-4a5a-9523-d1489589567e admin admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:38:32.763926 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff61ef1f-ff78-47c0-90be-2b4505f1b614 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:38:32.764902 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff61ef1f-ff78-47c0-90be-2b4505f1b614 admin admin] REQUEST_METHOD: `PUT` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:38:32.765236 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff61ef1f-ff78-47c0-90be-2b4505f1b614 admin admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:38:32.765949 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff61ef1f-ff78-47c0-90be-2b4505f1b614 admin admin] PATH_INFO: `/v3/projects/7916aa3905ba488089dde2d9099f1c2e/users/a9611894020a47008f3065789dd690cf/roles/6876e57cc51c448097ab109d35b2a3da` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:38:32.794154 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff61ef1f-ff78-47c0-90be-2b4505f1b614 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7916aa3905ba488089dde2d9099f1c2e, user_id=a9611894020a47008f3065789dd690cf, role_id=6876e57cc51c448097ab109d35b2a3da)` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:38:32.797138 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff61ef1f-ff78-47c0-90be-2b4505f1b614 admin admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:38:32.822435 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1283/2559] 10.222.0.49 () {60 vars in 1482 bytes} [Wed Jun 15 10:38:32 2022] PUT /identity/v3/projects/7916aa3905ba488089dde2d9099f1c2e/users/a9611894020a47008f3065789dd690cf/roles/6876e57cc51c448097ab109d35b2a3da => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:38:32.829709 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b1d9d9-a937-49bd-9537-c06fa584a520 None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:38:32.830211 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b1d9d9-a937-49bd-9537-c06fa584a520 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:38:32.830322 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b1d9d9-a937-49bd-9537-c06fa584a520 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:38:32.858401 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-89b1d9d9-a937-49bd-9537-c06fa584a520 None None] MFA Rules not processed for user `a9611894020a47008f3065789dd690cf`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:38:32.859732 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-89b1d9d9-a937-49bd-9537-c06fa584a520 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:38:32.885226 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:38:32.939910 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1277/2560] 10.222.0.49 () {58 vars in 957 bytes} [Wed Jun 15 10:38:32 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:38:32.944412 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff61ef1f-ff78-47c0-90be-2b4505f1b614 admin admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:38:33.020723 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff61ef1f-ff78-47c0-90be-2b4505f1b614 admin admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:38:33.028327 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd33a5e-d42e-4909-a259-baec5c009578 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:38:33.029010 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd33a5e-d42e-4909-a259-baec5c009578 admin admin] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:38:33.029304 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd33a5e-d42e-4909-a259-baec5c009578 admin admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:38:33.029577 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd33a5e-d42e-4909-a259-baec5c009578 admin admin] PATH_INFO: `/v3/projects` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:38:33.030286 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd33a5e-d42e-4909-a259-baec5c009578 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:38:33.032585 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd33a5e-d42e-4909-a259-baec5c009578 admin admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:38:33.070300 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1284/2561] 10.222.0.49 () {60 vars in 1152 bytes} [Wed Jun 15 10:38:32 2022] POST /identity/v3/projects => generated 393 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 10:38:33.071142 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b1d9d9-a937-49bd-9537-c06fa584a520 None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:38:33.087361 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b1d9d9-a937-49bd-9537-c06fa584a520 None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:38:33.148867 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f859c3c8-c5b6-4f77-8249-ff8833c00b38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:38:33.149806 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f859c3c8-c5b6-4f77-8249-ff8833c00b38 admin admin] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:38:33.150090 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f859c3c8-c5b6-4f77-8249-ff8833c00b38 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:38:33.150365 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f859c3c8-c5b6-4f77-8249-ff8833c00b38 admin admin] PATH_INFO: `/v3/users` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:38:33.151063 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f859c3c8-c5b6-4f77-8249-ff8833c00b38 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:38:33.153012 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f859c3c8-c5b6-4f77-8249-ff8833c00b38 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:38:33.174951 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1278/2562] 10.222.0.49 () {60 vars in 1143 bytes} [Wed Jun 15 10:38:33 2022] POST /identity/v3/users => generated 303 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 10:38:33.178837 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd33a5e-d42e-4909-a259-baec5c009578 admin admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:38:33.190496 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd33a5e-d42e-4909-a259-baec5c009578 admin admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:38:33.195627 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09062a6-b472-471a-94ff-eea0ab8a4b0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:38:33.196381 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09062a6-b472-471a-94ff-eea0ab8a4b0c admin admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:38:33.196673 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09062a6-b472-471a-94ff-eea0ab8a4b0c admin admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:38:33.196951 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09062a6-b472-471a-94ff-eea0ab8a4b0c admin admin] PATH_INFO: `/v3/roles` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:38:33.197718 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09062a6-b472-471a-94ff-eea0ab8a4b0c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:38:33.199366 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09062a6-b472-471a-94ff-eea0ab8a4b0c admin admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:38:33.212144 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1285/2563] 10.222.0.49 () {58 vars in 1121 bytes} [Wed Jun 15 10:38:33 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 10:38:33.215532 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f859c3c8-c5b6-4f77-8249-ff8833c00b38 admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:38:33.227842 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f859c3c8-c5b6-4f77-8249-ff8833c00b38 admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:38:33.233318 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95cce13f-9bd0-4fd5-8d56-bd9d9d4007c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:38:33.234329 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cce13f-9bd0-4fd5-8d56-bd9d9d4007c4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:38:33.234716 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cce13f-9bd0-4fd5-8d56-bd9d9d4007c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:38:33.235150 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cce13f-9bd0-4fd5-8d56-bd9d9d4007c4 admin admin] PATH_INFO: `/v3/projects/5bde1122d78b42b3a44362c818ea9cae/users/b580057e18d34d929df41ef8f21ed67a/roles/8051174e7a7947d5b991ebd793c2e543` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:38:33.252549 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95cce13f-9bd0-4fd5-8d56-bd9d9d4007c4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5bde1122d78b42b3a44362c818ea9cae, user_id=b580057e18d34d929df41ef8f21ed67a, role_id=8051174e7a7947d5b991ebd793c2e543)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:38:33.255165 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95cce13f-9bd0-4fd5-8d56-bd9d9d4007c4 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:38:33.271688 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1279/2564] 10.222.0.49 () {60 vars in 1482 bytes} [Wed Jun 15 10:38:33 2022] PUT /identity/v3/projects/5bde1122d78b42b3a44362c818ea9cae/users/b580057e18d34d929df41ef8f21ed67a/roles/8051174e7a7947d5b991ebd793c2e543 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:38:33.275938 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09062a6-b472-471a-94ff-eea0ab8a4b0c admin admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:38:33.286008 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09062a6-b472-471a-94ff-eea0ab8a4b0c admin admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:38:33.345359 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=93,keystone:INSERT=4,keystone:DELETE=3 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:38:33.365561 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3720bd-8df6-4f55-8c29-86d803815742 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:38:33.366517 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3720bd-8df6-4f55-8c29-86d803815742 admin admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:38:33.366958 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3720bd-8df6-4f55-8c29-86d803815742 admin admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:38:33.367417 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3720bd-8df6-4f55-8c29-86d803815742 admin admin] PATH_INFO: `/v3/roles` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:38:33.368628 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3720bd-8df6-4f55-8c29-86d803815742 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:38:33.371244 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3720bd-8df6-4f55-8c29-86d803815742 admin admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:38:33.386667 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1286/2565] 10.222.0.49 () {58 vars in 1121 bytes} [Wed Jun 15 10:38:33 2022] GET /identity/v3/roles => generated 1420 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 10:38:33.387544 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95cce13f-9bd0-4fd5-8d56-bd9d9d4007c4 admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:38:33.401478 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95cce13f-9bd0-4fd5-8d56-bd9d9d4007c4 admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:38:33.407275 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023a8aa0-384b-487d-9bc4-347def325c14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:38:33.408557 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023a8aa0-384b-487d-9bc4-347def325c14 admin admin] REQUEST_METHOD: `PUT` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:38:33.409121 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023a8aa0-384b-487d-9bc4-347def325c14 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:38:33.409406 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023a8aa0-384b-487d-9bc4-347def325c14 admin admin] PATH_INFO: `/v3/projects/5bde1122d78b42b3a44362c818ea9cae/users/b580057e18d34d929df41ef8f21ed67a/roles/8051174e7a7947d5b991ebd793c2e543` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:38:33.411692 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-023a8aa0-384b-487d-9bc4-347def325c14 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5bde1122d78b42b3a44362c818ea9cae, user_id=b580057e18d34d929df41ef8f21ed67a, role_id=8051174e7a7947d5b991ebd793c2e543)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:38:33.423148 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-023a8aa0-384b-487d-9bc4-347def325c14 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:38:33.426817 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1280/2566] 10.222.0.49 () {60 vars in 1482 bytes} [Wed Jun 15 10:38:33 2022] PUT /identity/v3/projects/5bde1122d78b42b3a44362c818ea9cae/users/b580057e18d34d929df41ef8f21ed67a/roles/8051174e7a7947d5b991ebd793c2e543 => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 10:38:33.439083 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73cb828-46aa-41dc-b385-8dabaa503847 None None] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:38:33.439413 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73cb828-46aa-41dc-b385-8dabaa503847 None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:38:33.439752 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73cb828-46aa-41dc-b385-8dabaa503847 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:38:33.469753 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.auth.core [None req-b73cb828-46aa-41dc-b385-8dabaa503847 None None] MFA Rules not processed for user `b580057e18d34d929df41ef8f21ed67a`. Rule list: `[]` (Enabled: `True`). {{(pid=80119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:38:33.471403 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-b73cb828-46aa-41dc-b385-8dabaa503847 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:38:33.570928 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1287/2567] 10.222.0.49 () {58 vars in 957 bytes} [Wed Jun 15 10:38:33 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:38:33.584128 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023a8aa0-384b-487d-9bc4-347def325c14 admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:38:33.585916 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-023a8aa0-384b-487d-9bc4-347def325c14 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:38:33.690069 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023a8aa0-384b-487d-9bc4-347def325c14 admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:38:33.693964 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2bcc27d-dea0-4427-a1f7-9c31057e8c30 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20d1327553404b80aecee87c78fcb52b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '559b3e713f5844b6ae37151d56dd419f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:38:33.695020 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bcc27d-dea0-4427-a1f7-9c31057e8c30 service cinder] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:38:33.695354 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bcc27d-dea0-4427-a1f7-9c31057e8c30 service cinder] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:38:33.696284 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bcc27d-dea0-4427-a1f7-9c31057e8c30 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:38:33.708940 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2bcc27d-dea0-4427-a1f7-9c31057e8c30 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:38:33.710432 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2bcc27d-dea0-4427-a1f7-9c31057e8c30 service cinder] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:38:33.724019 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1281/2568] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Jun 15 10:38:33 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:38:33.782394 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73cb828-46aa-41dc-b385-8dabaa503847 None None] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:38:33.784097 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-b73cb828-46aa-41dc-b385-8dabaa503847 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:38:33.805179 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73cb828-46aa-41dc-b385-8dabaa503847 None None] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:38:33.808824 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020fa456-6782-45e3-b227-bc75986d3535 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20d1327553404b80aecee87c78fcb52b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '559b3e713f5844b6ae37151d56dd419f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:38:33.809508 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020fa456-6782-45e3-b227-bc75986d3535 service cinder] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:38:33.809794 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020fa456-6782-45e3-b227-bc75986d3535 service cinder] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:38:33.810069 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020fa456-6782-45e3-b227-bc75986d3535 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:38:33.876537 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-020fa456-6782-45e3-b227-bc75986d3535 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:38:33.878042 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-020fa456-6782-45e3-b227-bc75986d3535 service cinder] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:38:33.887924 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1288/2569] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Jun 15 10:38:33 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:38:34.303973 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2bcc27d-dea0-4427-a1f7-9c31057e8c30 service cinder] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:38:34.315186 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2bcc27d-dea0-4427-a1f7-9c31057e8c30 service cinder] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:38:34.321277 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b91bdb2-b25e-43c0-80eb-a3c78bb0ba8e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20d1327553404b80aecee87c78fcb52b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '559b3e713f5844b6ae37151d56dd419f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:38:34.321945 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b91bdb2-b25e-43c0-80eb-a3c78bb0ba8e service cinder] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:38:34.322236 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b91bdb2-b25e-43c0-80eb-a3c78bb0ba8e service cinder] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:38:34.322535 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b91bdb2-b25e-43c0-80eb-a3c78bb0ba8e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:38:34.324359 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-9b91bdb2-b25e-43c0-80eb-a3c78bb0ba8e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:38:34.335035 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:38:34.397279 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b91bdb2-b25e-43c0-80eb-a3c78bb0ba8e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:38:34.398703 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b91bdb2-b25e-43c0-80eb-a3c78bb0ba8e service cinder] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:38:34.408795 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1282/2570] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Jun 15 10:38:34 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:38:35.180343 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020fa456-6782-45e3-b227-bc75986d3535 service cinder] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:38:35.182362 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-020fa456-6782-45e3-b227-bc75986d3535 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:38:35.241420 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b91bdb2-b25e-43c0-80eb-a3c78bb0ba8e service cinder] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:38:35.242753 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-9b91bdb2-b25e-43c0-80eb-a3c78bb0ba8e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:38:35.262964 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020fa456-6782-45e3-b227-bc75986d3535 service cinder] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:38:35.268308 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c60e692-ef8c-4866-965b-44df8cd484ff service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd03612a525644518517069b3fa43522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '559b3e713f5844b6ae37151d56dd419f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:38:35.268972 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c60e692-ef8c-4866-965b-44df8cd484ff service placement] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:38:35.269280 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c60e692-ef8c-4866-965b-44df8cd484ff service placement] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:38:35.269537 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c60e692-ef8c-4866-965b-44df8cd484ff service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:38:35.272816 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-1c60e692-ef8c-4866-965b-44df8cd484ff service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:38:35.283435 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=100 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:38:35.286049 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c60e692-ef8c-4866-965b-44df8cd484ff service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:38:35.289642 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c60e692-ef8c-4866-965b-44df8cd484ff service placement] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:38:35.303599 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1289/2571] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Jun 15 10:38:35 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:38:35.347763 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b91bdb2-b25e-43c0-80eb-a3c78bb0ba8e service cinder] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:38:35.353079 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3cb67fe-0f86-4bf7-b6c6-70b2d2efbcd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:38:35.354035 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cb67fe-0f86-4bf7-b6c6-70b2d2efbcd8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:38:35.354332 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cb67fe-0f86-4bf7-b6c6-70b2d2efbcd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:38:35.354672 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cb67fe-0f86-4bf7-b6c6-70b2d2efbcd8 admin admin] PATH_INFO: `/v3/users/cca7cb5e7620446ebae91b1fe12bbd24` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:38:35.355904 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3cb67fe-0f86-4bf7-b6c6-70b2d2efbcd8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cca7cb5e7620446ebae91b1fe12bbd24)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:38:35.357955 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3cb67fe-0f86-4bf7-b6c6-70b2d2efbcd8 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:38:35.408901 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-a3cb67fe-0f86-4bf7-b6c6-70b2d2efbcd8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cca7cb5e7620446ebae91b1fe12bbd24'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:38:35.416738 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-a3cb67fe-0f86-4bf7-b6c6-70b2d2efbcd8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cca7cb5e7620446ebae91b1fe12bbd24'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:38:35.417419 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-a3cb67fe-0f86-4bf7-b6c6-70b2d2efbcd8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cca7cb5e7620446ebae91b1fe12bbd24'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:38:35.418174 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-a3cb67fe-0f86-4bf7-b6c6-70b2d2efbcd8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cca7cb5e7620446ebae91b1fe12bbd24'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:38:35.425901 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-a3cb67fe-0f86-4bf7-b6c6-70b2d2efbcd8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cca7cb5e7620446ebae91b1fe12bbd24'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:38:35.444474 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1283/2572] 10.222.0.49 () {58 vars in 1223 bytes} [Wed Jun 15 10:38:35 2022] DELETE /identity/v3/users/cca7cb5e7620446ebae91b1fe12bbd24 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:38:35.622173 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c60e692-ef8c-4866-965b-44df8cd484ff service placement] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:38:35.635820 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c60e692-ef8c-4866-965b-44df8cd484ff service placement] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:38:35.698323 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e1071c-2ff1-4055-b1d6-b66a9acc95ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:38:35.699278 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e1071c-2ff1-4055-b1d6-b66a9acc95ed admin admin] REQUEST_METHOD: `DELETE` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:38:35.699560 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e1071c-2ff1-4055-b1d6-b66a9acc95ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:38:35.699829 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e1071c-2ff1-4055-b1d6-b66a9acc95ed admin admin] PATH_INFO: `/v3/projects/5dc28af73e82414d8f709e447286a962` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:38:35.701014 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e1071c-2ff1-4055-b1d6-b66a9acc95ed admin admin] RBAC: Authorizing `identity:delete_project(project_id=5dc28af73e82414d8f709e447286a962)` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:38:35.702937 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e1071c-2ff1-4055-b1d6-b66a9acc95ed admin admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:38:35.767070 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-95e1071c-2ff1-4055-b1d6-b66a9acc95ed admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5dc28af73e82414d8f709e447286a962', 'request_id': 'req-95e1071c-2ff1-4055-b1d6-b66a9acc95ed'} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:38:35.782432 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-95e1071c-2ff1-4055-b1d6-b66a9acc95ed admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5dc28af73e82414d8f709e447286a962', 'request_id': 'req-95e1071c-2ff1-4055-b1d6-b66a9acc95ed'} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:38:35.794733 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-95e1071c-2ff1-4055-b1d6-b66a9acc95ed admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5dc28af73e82414d8f709e447286a962', 'request_id': 'req-95e1071c-2ff1-4055-b1d6-b66a9acc95ed'} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:38:35.815728 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-95e1071c-2ff1-4055-b1d6-b66a9acc95ed admin admin] The token cache is being invalidate because project 5dc28af73e82414d8f709e447286a962 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80119) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 10:38:35.816023 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-95e1071c-2ff1-4055-b1d6-b66a9acc95ed admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:38:35.816708 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-95e1071c-2ff1-4055-b1d6-b66a9acc95ed admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:38:35.819348 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1290/2573] 10.222.0.49 () {58 vars in 1232 bytes} [Wed Jun 15 10:38:35 2022] DELETE /identity/v3/projects/5dc28af73e82414d8f709e447286a962 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:38:35.824107 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb62eae-4bff-49da-82d7-3f381765aa58 None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:38:35.824661 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb62eae-4bff-49da-82d7-3f381765aa58 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:38:35.824944 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb62eae-4bff-49da-82d7-3f381765aa58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:38:35.845449 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-efb62eae-4bff-49da-82d7-3f381765aa58 None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:38:35.846930 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-efb62eae-4bff-49da-82d7-3f381765aa58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:38:35.878538 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:38:35.935638 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1284/2574] 10.222.0.49 () {58 vars in 957 bytes} [Wed Jun 15 10:38:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:38:35.944334 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41682318-8911-4351-98eb-30a783b7c4f8 None None] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:38:35.945192 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41682318-8911-4351-98eb-30a783b7c4f8 None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:38:35.945797 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41682318-8911-4351-98eb-30a783b7c4f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:38:35.972567 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.auth.core [None req-41682318-8911-4351-98eb-30a783b7c4f8 None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:38:35.973806 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-41682318-8911-4351-98eb-30a783b7c4f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:38:35.983027 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1291/2575] 10.222.0.49 () {58 vars in 957 bytes} [Wed Jun 15 10:38:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:38:35.987891 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb62eae-4bff-49da-82d7-3f381765aa58 None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:38:35.998420 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb62eae-4bff-49da-82d7-3f381765aa58 None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:38:36.003386 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a065d03-39fc-4847-a1f7-092a5f7cc0e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:38:36.004139 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a065d03-39fc-4847-a1f7-092a5f7cc0e8 admin admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:38:36.004790 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a065d03-39fc-4847-a1f7-092a5f7cc0e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:38:36.005136 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a065d03-39fc-4847-a1f7-092a5f7cc0e8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:38:36.005982 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a065d03-39fc-4847-a1f7-092a5f7cc0e8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:38:36.006358 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a065d03-39fc-4847-a1f7-092a5f7cc0e8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:38:36.008952 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a065d03-39fc-4847-a1f7-092a5f7cc0e8 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:38:36.022920 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1285/2576] 10.222.0.49 () {58 vars in 1152 bytes} [Wed Jun 15 10:38:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:38:36.028592 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41682318-8911-4351-98eb-30a783b7c4f8 None None] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:38:36.043523 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41682318-8911-4351-98eb-30a783b7c4f8 None None] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:38:36.051859 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a320cd30-e8ba-4166-b013-dcd1cfcb3dca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:38:36.052791 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a320cd30-e8ba-4166-b013-dcd1cfcb3dca admin admin] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:38:36.053252 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a320cd30-e8ba-4166-b013-dcd1cfcb3dca admin admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:38:36.053571 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a320cd30-e8ba-4166-b013-dcd1cfcb3dca admin admin] PATH_INFO: `/v3/projects` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:38:36.054518 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a320cd30-e8ba-4166-b013-dcd1cfcb3dca admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:38:36.056983 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a320cd30-e8ba-4166-b013-dcd1cfcb3dca admin admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:38:36.084761 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1292/2577] 10.222.0.49 () {60 vars in 1152 bytes} [Wed Jun 15 10:38:36 2022] POST /identity/v3/projects => generated 369 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 10:38:36.089474 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a065d03-39fc-4847-a1f7-092a5f7cc0e8 admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:38:36.102972 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a065d03-39fc-4847-a1f7-092a5f7cc0e8 admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:38:36.174699 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8168d43d-3ad2-4491-b6f5-49b5b012af51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:38:36.175405 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8168d43d-3ad2-4491-b6f5-49b5b012af51 admin admin] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:38:36.175809 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8168d43d-3ad2-4491-b6f5-49b5b012af51 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:38:36.176102 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8168d43d-3ad2-4491-b6f5-49b5b012af51 admin admin] PATH_INFO: `/v3/users` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:38:36.176792 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8168d43d-3ad2-4491-b6f5-49b5b012af51 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:38:36.178694 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8168d43d-3ad2-4491-b6f5-49b5b012af51 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:38:36.198345 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1286/2578] 10.222.0.49 () {60 vars in 1143 bytes} [Wed Jun 15 10:38:36 2022] POST /identity/v3/users => generated 285 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 10:38:36.202466 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a320cd30-e8ba-4166-b013-dcd1cfcb3dca admin admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:38:36.213660 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a320cd30-e8ba-4166-b013-dcd1cfcb3dca admin admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:38:36.218897 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22c10ea-b737-45c4-8933-2241531b8599 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:38:36.220292 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22c10ea-b737-45c4-8933-2241531b8599 admin admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:38:36.220582 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22c10ea-b737-45c4-8933-2241531b8599 admin admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:38:36.220860 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22c10ea-b737-45c4-8933-2241531b8599 admin admin] PATH_INFO: `/v3/roles` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:38:36.221584 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f22c10ea-b737-45c4-8933-2241531b8599 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:38:36.223237 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f22c10ea-b737-45c4-8933-2241531b8599 admin admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:38:36.233832 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1293/2579] 10.222.0.49 () {58 vars in 1121 bytes} [Wed Jun 15 10:38:36 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 10:38:36.237586 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8168d43d-3ad2-4491-b6f5-49b5b012af51 admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:38:36.249255 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8168d43d-3ad2-4491-b6f5-49b5b012af51 admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:38:36.253953 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c02bc7-bfd2-4944-9782-0bddc26f4286 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:38:36.254638 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c02bc7-bfd2-4944-9782-0bddc26f4286 admin admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:38:36.254923 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c02bc7-bfd2-4944-9782-0bddc26f4286 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:38:36.255235 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c02bc7-bfd2-4944-9782-0bddc26f4286 admin admin] PATH_INFO: `/v3/roles` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:38:36.255967 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c02bc7-bfd2-4944-9782-0bddc26f4286 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:38:36.257550 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c02bc7-bfd2-4944-9782-0bddc26f4286 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:38:36.268122 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1287/2580] 10.222.0.49 () {58 vars in 1121 bytes} [Wed Jun 15 10:38:36 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 10:38:36.272096 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22c10ea-b737-45c4-8933-2241531b8599 admin admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:38:36.282845 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22c10ea-b737-45c4-8933-2241531b8599 admin admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:38:36.287484 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14a7897-ac87-4169-95ce-b2278287516d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:38:36.288384 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14a7897-ac87-4169-95ce-b2278287516d admin admin] REQUEST_METHOD: `PUT` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:38:36.288670 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14a7897-ac87-4169-95ce-b2278287516d admin admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:38:36.288958 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14a7897-ac87-4169-95ce-b2278287516d admin admin] PATH_INFO: `/v3/projects/4fc87baa62314a1ca6fd85e95e3b5782/users/1366259de7164616a1da93ddb176e51c/roles/6876e57cc51c448097ab109d35b2a3da` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:38:36.303838 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d14a7897-ac87-4169-95ce-b2278287516d admin admin] RBAC: Authorizing `identity:create_grant(project_id=4fc87baa62314a1ca6fd85e95e3b5782, user_id=1366259de7164616a1da93ddb176e51c, role_id=6876e57cc51c448097ab109d35b2a3da)` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:38:36.306818 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d14a7897-ac87-4169-95ce-b2278287516d admin admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:38:36.319286 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1294/2581] 10.222.0.49 () {60 vars in 1482 bytes} [Wed Jun 15 10:38:36 2022] PUT /identity/v3/projects/4fc87baa62314a1ca6fd85e95e3b5782/users/1366259de7164616a1da93ddb176e51c/roles/6876e57cc51c448097ab109d35b2a3da => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:38:36.331724 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf89205c-6ca5-47b9-ab0c-a130fb70bb16 None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:38:36.332087 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf89205c-6ca5-47b9-ab0c-a130fb70bb16 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:38:36.332461 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf89205c-6ca5-47b9-ab0c-a130fb70bb16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:38:36.358960 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-bf89205c-6ca5-47b9-ab0c-a130fb70bb16 None None] MFA Rules not processed for user `1366259de7164616a1da93ddb176e51c`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:38:36.360208 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-bf89205c-6ca5-47b9-ab0c-a130fb70bb16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:38:36.425966 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1288/2582] 10.222.0.49 () {58 vars in 957 bytes} [Wed Jun 15 10:38:36 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:38:36.430309 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14a7897-ac87-4169-95ce-b2278287516d admin admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:38:36.457175 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:38:36.495379 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14a7897-ac87-4169-95ce-b2278287516d admin admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:38:36.500086 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e2af0e-2901-4caf-85f8-a023037b5a68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:38:36.500743 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e2af0e-2901-4caf-85f8-a023037b5a68 admin admin] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:38:36.501029 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e2af0e-2901-4caf-85f8-a023037b5a68 admin admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:38:36.501303 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e2af0e-2901-4caf-85f8-a023037b5a68 admin admin] PATH_INFO: `/v3/projects` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:38:36.501991 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e2af0e-2901-4caf-85f8-a023037b5a68 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:38:36.503924 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e2af0e-2901-4caf-85f8-a023037b5a68 admin admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:38:36.528544 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1295/2583] 10.222.0.49 () {60 vars in 1152 bytes} [Wed Jun 15 10:38:36 2022] POST /identity/v3/projects => generated 369 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 10:38:36.533599 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf89205c-6ca5-47b9-ab0c-a130fb70bb16 None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:38:36.546789 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf89205c-6ca5-47b9-ab0c-a130fb70bb16 None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:38:36.565610 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:38:36.604803 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25813759-af76-43b6-83f1-bd7bc76afe6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:38:36.605473 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25813759-af76-43b6-83f1-bd7bc76afe6f admin admin] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:38:36.605757 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25813759-af76-43b6-83f1-bd7bc76afe6f admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:38:36.606030 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25813759-af76-43b6-83f1-bd7bc76afe6f admin admin] PATH_INFO: `/v3/users` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:38:36.606708 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25813759-af76-43b6-83f1-bd7bc76afe6f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:38:36.608650 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25813759-af76-43b6-83f1-bd7bc76afe6f admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:38:36.628506 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1289/2584] 10.222.0.49 () {60 vars in 1143 bytes} [Wed Jun 15 10:38:36 2022] POST /identity/v3/users => generated 291 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 10:38:36.633194 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e2af0e-2901-4caf-85f8-a023037b5a68 admin admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:38:36.644581 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e2af0e-2901-4caf-85f8-a023037b5a68 admin admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:38:36.650215 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ef0a4a-6ec8-4e36-a49d-3056a84a43c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:38:36.650949 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ef0a4a-6ec8-4e36-a49d-3056a84a43c0 admin admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:38:36.651270 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ef0a4a-6ec8-4e36-a49d-3056a84a43c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:38:36.651549 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ef0a4a-6ec8-4e36-a49d-3056a84a43c0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:38:36.652268 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ef0a4a-6ec8-4e36-a49d-3056a84a43c0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:38:36.653863 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ef0a4a-6ec8-4e36-a49d-3056a84a43c0 admin admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:38:36.664593 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1296/2585] 10.222.0.49 () {58 vars in 1121 bytes} [Wed Jun 15 10:38:36 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 10:38:36.669079 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25813759-af76-43b6-83f1-bd7bc76afe6f admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:38:36.680200 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25813759-af76-43b6-83f1-bd7bc76afe6f admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:38:36.684870 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac723f6-8b22-4ed8-895f-f469f1a87a25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:38:36.685793 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac723f6-8b22-4ed8-895f-f469f1a87a25 admin admin] REQUEST_METHOD: `PUT` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:38:36.686082 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac723f6-8b22-4ed8-895f-f469f1a87a25 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:38:36.686359 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac723f6-8b22-4ed8-895f-f469f1a87a25 admin admin] PATH_INFO: `/v3/projects/30ed0e4bace545d797e93ac15b5ad45a/users/5b939993075c442192cad4befc704b00/roles/8051174e7a7947d5b991ebd793c2e543` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:38:36.701090 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac723f6-8b22-4ed8-895f-f469f1a87a25 admin admin] RBAC: Authorizing `identity:create_grant(project_id=30ed0e4bace545d797e93ac15b5ad45a, user_id=5b939993075c442192cad4befc704b00, role_id=8051174e7a7947d5b991ebd793c2e543)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:38:36.703779 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac723f6-8b22-4ed8-895f-f469f1a87a25 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:38:36.717187 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1290/2586] 10.222.0.49 () {60 vars in 1482 bytes} [Wed Jun 15 10:38:36 2022] PUT /identity/v3/projects/30ed0e4bace545d797e93ac15b5ad45a/users/5b939993075c442192cad4befc704b00/roles/8051174e7a7947d5b991ebd793c2e543 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:38:36.721543 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ef0a4a-6ec8-4e36-a49d-3056a84a43c0 admin admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:38:36.732115 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ef0a4a-6ec8-4e36-a49d-3056a84a43c0 admin admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:38:36.790904 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87f7874-2d05-4686-8a05-61cf780032b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:38:36.791694 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87f7874-2d05-4686-8a05-61cf780032b4 admin admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:38:36.792049 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87f7874-2d05-4686-8a05-61cf780032b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:38:36.792392 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87f7874-2d05-4686-8a05-61cf780032b4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:38:36.793216 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f87f7874-2d05-4686-8a05-61cf780032b4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:38:36.794924 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f87f7874-2d05-4686-8a05-61cf780032b4 admin admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:38:36.805516 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1297/2587] 10.222.0.49 () {58 vars in 1121 bytes} [Wed Jun 15 10:38:36 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 10:38:36.809806 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac723f6-8b22-4ed8-895f-f469f1a87a25 admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:38:36.820137 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac723f6-8b22-4ed8-895f-f469f1a87a25 admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:38:36.825136 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b45365-7abd-48fa-be78-8f9d7db8f6c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:38:36.826023 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b45365-7abd-48fa-be78-8f9d7db8f6c3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:38:36.826300 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b45365-7abd-48fa-be78-8f9d7db8f6c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:38:36.826620 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b45365-7abd-48fa-be78-8f9d7db8f6c3 admin admin] PATH_INFO: `/v3/projects/30ed0e4bace545d797e93ac15b5ad45a/users/5b939993075c442192cad4befc704b00/roles/8051174e7a7947d5b991ebd793c2e543` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:38:36.828730 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b45365-7abd-48fa-be78-8f9d7db8f6c3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=30ed0e4bace545d797e93ac15b5ad45a, user_id=5b939993075c442192cad4befc704b00, role_id=8051174e7a7947d5b991ebd793c2e543)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:38:36.831566 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b45365-7abd-48fa-be78-8f9d7db8f6c3 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:38:36.841825 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1291/2588] 10.222.0.49 () {60 vars in 1482 bytes} [Wed Jun 15 10:38:36 2022] PUT /identity/v3/projects/30ed0e4bace545d797e93ac15b5ad45a/users/5b939993075c442192cad4befc704b00/roles/8051174e7a7947d5b991ebd793c2e543 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:38:36.852648 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba4de2c-71b4-473e-9d5a-632dee01a475 None None] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:38:36.852946 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba4de2c-71b4-473e-9d5a-632dee01a475 None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:38:36.853229 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba4de2c-71b4-473e-9d5a-632dee01a475 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:38:36.882375 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.auth.core [None req-6ba4de2c-71b4-473e-9d5a-632dee01a475 None None] MFA Rules not processed for user `5b939993075c442192cad4befc704b00`. Rule list: `[]` (Enabled: `True`). {{(pid=80119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:38:36.883719 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-6ba4de2c-71b4-473e-9d5a-632dee01a475 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:38:36.951604 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1298/2589] 10.222.0.49 () {58 vars in 957 bytes} [Wed Jun 15 10:38:36 2022] POST /identity/v3/auth/tokens => generated 3303 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:38:36.963288 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b45365-7abd-48fa-be78-8f9d7db8f6c3 admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:38:36.965022 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-f3b45365-7abd-48fa-be78-8f9d7db8f6c3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:38:36.999047 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba4de2c-71b4-473e-9d5a-632dee01a475 None None] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:38:37.000480 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-6ba4de2c-71b4-473e-9d5a-632dee01a475 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:38:37.022354 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:38:37.023609 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b45365-7abd-48fa-be78-8f9d7db8f6c3 admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:38:37.027621 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe3b5fd-c9f7-4e21-abc8-473d32c354bd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20d1327553404b80aecee87c78fcb52b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '559b3e713f5844b6ae37151d56dd419f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:38:37.028271 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe3b5fd-c9f7-4e21-abc8-473d32c354bd service cinder] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:38:37.028555 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe3b5fd-c9f7-4e21-abc8-473d32c354bd service cinder] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:38:37.028835 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe3b5fd-c9f7-4e21-abc8-473d32c354bd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:38:37.039169 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe3b5fd-c9f7-4e21-abc8-473d32c354bd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:38:37.040612 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe3b5fd-c9f7-4e21-abc8-473d32c354bd service cinder] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:38:37.050582 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1292/2590] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Jun 15 10:38:36 2022] GET /identity/v3/auth/tokens => generated 3303 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:38:37.069328 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba4de2c-71b4-473e-9d5a-632dee01a475 None None] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:38:37.074977 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472b8b27-ebbf-4a47-8552-939baeff4ed1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:38:37.075887 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472b8b27-ebbf-4a47-8552-939baeff4ed1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:38:37.076167 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472b8b27-ebbf-4a47-8552-939baeff4ed1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:38:37.076440 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472b8b27-ebbf-4a47-8552-939baeff4ed1 admin admin] PATH_INFO: `/v3/users/80e35b835c1a4289956be479b35e3a31` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:38:37.077635 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-472b8b27-ebbf-4a47-8552-939baeff4ed1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=80e35b835c1a4289956be479b35e3a31)` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:38:37.080116 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-472b8b27-ebbf-4a47-8552-939baeff4ed1 admin admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:38:37.094273 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe3b5fd-c9f7-4e21-abc8-473d32c354bd service cinder] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:38:37.096348 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-ebe3b5fd-c9f7-4e21-abc8-473d32c354bd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:38:37.108701 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe3b5fd-c9f7-4e21-abc8-473d32c354bd service cinder] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:38:37.112956 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0641278-1d07-4f1c-aeba-f48535946cd3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20d1327553404b80aecee87c78fcb52b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '559b3e713f5844b6ae37151d56dd419f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:38:37.113828 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0641278-1d07-4f1c-aeba-f48535946cd3 service cinder] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:38:37.114124 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0641278-1d07-4f1c-aeba-f48535946cd3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:38:37.114408 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0641278-1d07-4f1c-aeba-f48535946cd3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:38:37.127260 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-472b8b27-ebbf-4a47-8552-939baeff4ed1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '80e35b835c1a4289956be479b35e3a31'} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:38:37.134171 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-472b8b27-ebbf-4a47-8552-939baeff4ed1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '80e35b835c1a4289956be479b35e3a31'} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:38:37.134857 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-472b8b27-ebbf-4a47-8552-939baeff4ed1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '80e35b835c1a4289956be479b35e3a31'} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:38:37.135502 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-472b8b27-ebbf-4a47-8552-939baeff4ed1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '80e35b835c1a4289956be479b35e3a31'} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:38:37.143410 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-472b8b27-ebbf-4a47-8552-939baeff4ed1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '80e35b835c1a4289956be479b35e3a31'} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:38:37.163239 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1299/2591] 10.222.0.49 () {58 vars in 1223 bytes} [Wed Jun 15 10:38:36 2022] DELETE /identity/v3/users/80e35b835c1a4289956be479b35e3a31 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 10:38:37.167677 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea124309-a0bf-4901-9768-625b424e9cd3 None None] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:38:37.168058 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea124309-a0bf-4901-9768-625b424e9cd3 None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:38:37.168635 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea124309-a0bf-4901-9768-625b424e9cd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:38:37.178616 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0641278-1d07-4f1c-aeba-f48535946cd3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:38:37.180449 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0641278-1d07-4f1c-aeba-f48535946cd3 service cinder] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:38:37.186739 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.auth.core [None req-ea124309-a0bf-4901-9768-625b424e9cd3 None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:38:37.188210 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-ea124309-a0bf-4901-9768-625b424e9cd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:38:37.190582 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1293/2592] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Jun 15 10:38:37 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 10:38:37.267184 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1300/2593] 10.222.0.49 () {58 vars in 957 bytes} [Wed Jun 15 10:38:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 10:38:37.280342 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0641278-1d07-4f1c-aeba-f48535946cd3 service cinder] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:38:37.282663 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-b0641278-1d07-4f1c-aeba-f48535946cd3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:38:37.309103 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:38:37.362897 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0641278-1d07-4f1c-aeba-f48535946cd3 service cinder] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:38:37.440223 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0042e835-b8cc-4492-a8b5-c8976f1d0a21 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0cb44a17d205431793e9a8791e477a32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '559b3e713f5844b6ae37151d56dd419f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:38:37.442062 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0042e835-b8cc-4492-a8b5-c8976f1d0a21 service neutron] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:38:37.442268 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0042e835-b8cc-4492-a8b5-c8976f1d0a21 service neutron] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:38:37.442551 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0042e835-b8cc-4492-a8b5-c8976f1d0a21 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:38:37.454590 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0042e835-b8cc-4492-a8b5-c8976f1d0a21 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:38:37.456229 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0042e835-b8cc-4492-a8b5-c8976f1d0a21 service neutron] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:38:37.474804 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1294/2594] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Jun 15 10:38:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 196 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 10:38:37.486928 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea124309-a0bf-4901-9768-625b424e9cd3 None None] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:38:37.488935 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-ea124309-a0bf-4901-9768-625b424e9cd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:38:37.503309 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea124309-a0bf-4901-9768-625b424e9cd3 None None] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:38:37.510545 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d80fe5f-5004-4639-8344-db9c9b5bcc50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:38:37.511617 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d80fe5f-5004-4639-8344-db9c9b5bcc50 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:38:37.511962 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d80fe5f-5004-4639-8344-db9c9b5bcc50 admin admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:38:37.512246 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d80fe5f-5004-4639-8344-db9c9b5bcc50 admin admin] PATH_INFO: `/v3/users/a9611894020a47008f3065789dd690cf` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:38:37.513536 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d80fe5f-5004-4639-8344-db9c9b5bcc50 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a9611894020a47008f3065789dd690cf)` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:38:37.515777 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d80fe5f-5004-4639-8344-db9c9b5bcc50 admin admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:38:37.563022 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:38:37.580433 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-5d80fe5f-5004-4639-8344-db9c9b5bcc50 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a9611894020a47008f3065789dd690cf'} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:38:37.591949 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-5d80fe5f-5004-4639-8344-db9c9b5bcc50 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a9611894020a47008f3065789dd690cf'} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:38:37.593107 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-5d80fe5f-5004-4639-8344-db9c9b5bcc50 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a9611894020a47008f3065789dd690cf'} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:38:37.593862 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-5d80fe5f-5004-4639-8344-db9c9b5bcc50 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a9611894020a47008f3065789dd690cf'} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:38:37.605932 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-5d80fe5f-5004-4639-8344-db9c9b5bcc50 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a9611894020a47008f3065789dd690cf'} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:38:37.624986 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1301/2595] 10.222.0.49 () {58 vars in 1223 bytes} [Wed Jun 15 10:38:37 2022] DELETE /identity/v3/users/a9611894020a47008f3065789dd690cf => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:38:37.630123 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80dfbd93-8ed1-45d4-8981-8f8db783e286 None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:38:37.631016 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80dfbd93-8ed1-45d4-8981-8f8db783e286 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:38:37.631555 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80dfbd93-8ed1-45d4-8981-8f8db783e286 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:38:37.654347 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-80dfbd93-8ed1-45d4-8981-8f8db783e286 None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:38:37.655713 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-80dfbd93-8ed1-45d4-8981-8f8db783e286 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:38:37.718305 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1295/2596] 10.222.0.49 () {58 vars in 957 bytes} [Wed Jun 15 10:38:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:38:37.729386 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d80fe5f-5004-4639-8344-db9c9b5bcc50 admin admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:38:37.731057 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-5d80fe5f-5004-4639-8344-db9c9b5bcc50 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:38:37.769572 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80dfbd93-8ed1-45d4-8981-8f8db783e286 None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:38:37.776636 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-80dfbd93-8ed1-45d4-8981-8f8db783e286 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:38:37.792787 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80dfbd93-8ed1-45d4-8981-8f8db783e286 None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:38:37.797856 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c72c7c-86d8-4b7d-b765-52ea87ae8ad9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:38:37.798743 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c72c7c-86d8-4b7d-b765-52ea87ae8ad9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:38:37.799172 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c72c7c-86d8-4b7d-b765-52ea87ae8ad9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:38:37.799444 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c72c7c-86d8-4b7d-b765-52ea87ae8ad9 admin admin] PATH_INFO: `/v3/projects/26798c32faa147409b464abe67438e4b` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:38:37.800847 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c72c7c-86d8-4b7d-b765-52ea87ae8ad9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=26798c32faa147409b464abe67438e4b)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:38:37.803133 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c72c7c-86d8-4b7d-b765-52ea87ae8ad9 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:38:37.810376 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d80fe5f-5004-4639-8344-db9c9b5bcc50 admin admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:38:37.814622 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c83351-a42c-4cb2-8cd4-5546d69317d3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0cb44a17d205431793e9a8791e477a32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '559b3e713f5844b6ae37151d56dd419f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:38:37.815346 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c83351-a42c-4cb2-8cd4-5546d69317d3 service neutron] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:38:37.815636 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c83351-a42c-4cb2-8cd4-5546d69317d3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:38:37.816507 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c83351-a42c-4cb2-8cd4-5546d69317d3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:38:37.829498 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c83351-a42c-4cb2-8cd4-5546d69317d3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:38:37.830961 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c83351-a42c-4cb2-8cd4-5546d69317d3 service neutron] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:38:37.842189 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1302/2597] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Jun 15 10:38:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:38:37.862224 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-82c72c7c-86d8-4b7d-b765-52ea87ae8ad9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '26798c32faa147409b464abe67438e4b', 'request_id': 'req-82c72c7c-86d8-4b7d-b765-52ea87ae8ad9'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:38:37.870100 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-82c72c7c-86d8-4b7d-b765-52ea87ae8ad9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '26798c32faa147409b464abe67438e4b', 'request_id': 'req-82c72c7c-86d8-4b7d-b765-52ea87ae8ad9'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:38:37.877191 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-82c72c7c-86d8-4b7d-b765-52ea87ae8ad9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '26798c32faa147409b464abe67438e4b', 'request_id': 'req-82c72c7c-86d8-4b7d-b765-52ea87ae8ad9'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:38:37.888158 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:38:37.894989 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-82c72c7c-86d8-4b7d-b765-52ea87ae8ad9 admin admin] The token cache is being invalidate because project 26798c32faa147409b464abe67438e4b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80120) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 10:38:37.895207 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-82c72c7c-86d8-4b7d-b765-52ea87ae8ad9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:38:37.896508 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-82c72c7c-86d8-4b7d-b765-52ea87ae8ad9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:38:37.901942 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1296/2598] 10.222.0.49 () {58 vars in 1232 bytes} [Wed Jun 15 10:38:37 2022] DELETE /identity/v3/projects/26798c32faa147409b464abe67438e4b => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 10:38:37.904012 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cd3ae1-01b8-4dc4-b413-11e29d00dea4 None None] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:38:37.904315 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cd3ae1-01b8-4dc4-b413-11e29d00dea4 None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:38:37.904596 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cd3ae1-01b8-4dc4-b413-11e29d00dea4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:38:37.925241 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.auth.core [None req-21cd3ae1-01b8-4dc4-b413-11e29d00dea4 None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:38:37.926521 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-21cd3ae1-01b8-4dc4-b413-11e29d00dea4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:38:37.986475 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1303/2599] 10.222.0.49 () {58 vars in 957 bytes} [Wed Jun 15 10:38:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:38:37.994226 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9bb784-33b5-40ec-b268-6737ba678020 None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:38:37.994572 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9bb784-33b5-40ec-b268-6737ba678020 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:38:37.994927 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9bb784-33b5-40ec-b268-6737ba678020 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:38:38.015053 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-9b9bb784-33b5-40ec-b268-6737ba678020 None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:38:38.016811 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-9b9bb784-33b5-40ec-b268-6737ba678020 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:38:38.025088 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1297/2600] 10.222.0.49 () {58 vars in 957 bytes} [Wed Jun 15 10:38:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:38:38.030479 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21cd3ae1-01b8-4dc4-b413-11e29d00dea4 None None] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:38:38.040655 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21cd3ae1-01b8-4dc4-b413-11e29d00dea4 None None] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:38:38.045379 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb70254-20be-493b-85df-c5b580ec7945 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:38:38.046038 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb70254-20be-493b-85df-c5b580ec7945 admin admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:38:38.046337 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb70254-20be-493b-85df-c5b580ec7945 admin admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:38:38.046590 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb70254-20be-493b-85df-c5b580ec7945 admin admin] PATH_INFO: `/v3/domains` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:38:38.047685 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb70254-20be-493b-85df-c5b580ec7945 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:38:38.047685 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb70254-20be-493b-85df-c5b580ec7945 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:38:38.049293 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb70254-20be-493b-85df-c5b580ec7945 admin admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:38:38.061298 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1304/2601] 10.222.0.49 () {58 vars in 1152 bytes} [Wed Jun 15 10:38:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:38:38.066273 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9bb784-33b5-40ec-b268-6737ba678020 None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:38:38.076568 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9bb784-33b5-40ec-b268-6737ba678020 None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:38:38.082418 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e633c1-886d-4816-8bf2-42d1ef7695c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:38:38.083234 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e633c1-886d-4816-8bf2-42d1ef7695c3 admin admin] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:38:38.083582 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e633c1-886d-4816-8bf2-42d1ef7695c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:38:38.083927 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e633c1-886d-4816-8bf2-42d1ef7695c3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:38:38.084710 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e633c1-886d-4816-8bf2-42d1ef7695c3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:38:38.086781 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e633c1-886d-4816-8bf2-42d1ef7695c3 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:38:38.112989 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1298/2602] 10.222.0.49 () {60 vars in 1152 bytes} [Wed Jun 15 10:38:38 2022] POST /identity/v3/projects => generated 379 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 10:38:38.117510 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb70254-20be-493b-85df-c5b580ec7945 admin admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:38:38.128855 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb70254-20be-493b-85df-c5b580ec7945 admin admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:38:38.133499 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e633c1-886d-4816-8bf2-42d1ef7695c3 admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:38:38.135659 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-87e633c1-886d-4816-8bf2-42d1ef7695c3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:38:38.178972 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:38:38.201602 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaac5a53-a92c-4069-a644-b294c46c5e45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:38:38.202290 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaac5a53-a92c-4069-a644-b294c46c5e45 admin admin] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:38:38.202582 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaac5a53-a92c-4069-a644-b294c46c5e45 admin admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:38:38.202896 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaac5a53-a92c-4069-a644-b294c46c5e45 admin admin] PATH_INFO: `/v3/users` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:38:38.203680 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaac5a53-a92c-4069-a644-b294c46c5e45 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:38:38.206269 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaac5a53-a92c-4069-a644-b294c46c5e45 admin admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:38:38.209251 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e633c1-886d-4816-8bf2-42d1ef7695c3 admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:38:38.213927 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b87945-356f-4617-8f6a-d2b61ff73f23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:38:38.214791 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b87945-356f-4617-8f6a-d2b61ff73f23 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:38:38.215076 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b87945-356f-4617-8f6a-d2b61ff73f23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:38:38.215380 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b87945-356f-4617-8f6a-d2b61ff73f23 admin admin] PATH_INFO: `/v3/projects/7916aa3905ba488089dde2d9099f1c2e` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:38:38.216567 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b87945-356f-4617-8f6a-d2b61ff73f23 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7916aa3905ba488089dde2d9099f1c2e)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:38:38.218468 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b87945-356f-4617-8f6a-d2b61ff73f23 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:38:38.226903 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1305/2603] 10.222.0.49 () {60 vars in 1143 bytes} [Wed Jun 15 10:38:38 2022] POST /identity/v3/users => generated 290 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 10:38:38.231007 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaac5a53-a92c-4069-a644-b294c46c5e45 admin admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:38:38.240695 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaac5a53-a92c-4069-a644-b294c46c5e45 admin admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:38:38.245530 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6b12f5-92ac-4974-8b81-8397003ff6f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:38:38.246261 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6b12f5-92ac-4974-8b81-8397003ff6f2 admin admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:38:38.246552 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6b12f5-92ac-4974-8b81-8397003ff6f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:38:38.246878 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6b12f5-92ac-4974-8b81-8397003ff6f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:38:38.247603 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca6b12f5-92ac-4974-8b81-8397003ff6f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:38:38.249344 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca6b12f5-92ac-4974-8b81-8397003ff6f2 admin admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:38:38.260863 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1306/2604] 10.222.0.49 () {58 vars in 1121 bytes} [Wed Jun 15 10:38:38 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 15 10:38:38.264802 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6b12f5-92ac-4974-8b81-8397003ff6f2 admin admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:38:38.265700 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-92b87945-356f-4617-8f6a-d2b61ff73f23 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7916aa3905ba488089dde2d9099f1c2e', 'request_id': 'req-92b87945-356f-4617-8f6a-d2b61ff73f23'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:38:38.274818 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-92b87945-356f-4617-8f6a-d2b61ff73f23 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7916aa3905ba488089dde2d9099f1c2e', 'request_id': 'req-92b87945-356f-4617-8f6a-d2b61ff73f23'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:38:38.275323 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6b12f5-92ac-4974-8b81-8397003ff6f2 admin admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:38:38.281594 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-92b87945-356f-4617-8f6a-d2b61ff73f23 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7916aa3905ba488089dde2d9099f1c2e', 'request_id': 'req-92b87945-356f-4617-8f6a-d2b61ff73f23'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:38:38.293285 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-92b87945-356f-4617-8f6a-d2b61ff73f23 admin admin] The token cache is being invalidate because project 7916aa3905ba488089dde2d9099f1c2e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80120) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 10:38:38.293578 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-92b87945-356f-4617-8f6a-d2b61ff73f23 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:38:38.294237 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-92b87945-356f-4617-8f6a-d2b61ff73f23 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:38:38.295828 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1299/2605] 10.222.0.49 () {58 vars in 1232 bytes} [Wed Jun 15 10:38:38 2022] DELETE /identity/v3/projects/7916aa3905ba488089dde2d9099f1c2e => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:38:38.301393 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b87945-356f-4617-8f6a-d2b61ff73f23 admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:38:38.302757 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-92b87945-356f-4617-8f6a-d2b61ff73f23 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:38:38.340148 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0121b50-958e-4327-8998-09fa3ed99668 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:38:38.340832 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0121b50-958e-4327-8998-09fa3ed99668 admin admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:38:38.341125 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0121b50-958e-4327-8998-09fa3ed99668 admin admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:38:38.341821 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0121b50-958e-4327-8998-09fa3ed99668 admin admin] PATH_INFO: `/v3/roles` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:38:38.342564 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0121b50-958e-4327-8998-09fa3ed99668 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:38:38.344209 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0121b50-958e-4327-8998-09fa3ed99668 admin admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:38:38.355304 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1307/2606] 10.222.0.49 () {58 vars in 1121 bytes} [Wed Jun 15 10:38:38 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 15 10:38:38.359647 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0121b50-958e-4327-8998-09fa3ed99668 admin admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:38:38.361443 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-e0121b50-958e-4327-8998-09fa3ed99668 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:38:38.374870 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0121b50-958e-4327-8998-09fa3ed99668 admin admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:38:38.380337 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1445680c-1b66-40cd-a07e-f55d1d2a146a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:38:38.381369 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1445680c-1b66-40cd-a07e-f55d1d2a146a admin admin] REQUEST_METHOD: `PUT` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:38:38.381642 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1445680c-1b66-40cd-a07e-f55d1d2a146a admin admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:38:38.381923 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1445680c-1b66-40cd-a07e-f55d1d2a146a admin admin] PATH_INFO: `/v3/projects/f096e399deac4d029ff63e3cb7f1ce9f/users/61bc9ad2c9444fc297eb4b5902a67b98/roles/6876e57cc51c448097ab109d35b2a3da` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:38:38.384900 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b87945-356f-4617-8f6a-d2b61ff73f23 admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:38:38.390309 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a466f672-e44e-4ac3-a59c-0d68668883f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:38:38.391828 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a466f672-e44e-4ac3-a59c-0d68668883f9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:38:38.392206 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a466f672-e44e-4ac3-a59c-0d68668883f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:38:38.392554 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a466f672-e44e-4ac3-a59c-0d68668883f9 admin admin] PATH_INFO: `/v3/users/b580057e18d34d929df41ef8f21ed67a` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:38:38.393792 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a466f672-e44e-4ac3-a59c-0d68668883f9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b580057e18d34d929df41ef8f21ed67a)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:38:38.395886 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a466f672-e44e-4ac3-a59c-0d68668883f9 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:38:38.397785 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:38:38.403408 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1445680c-1b66-40cd-a07e-f55d1d2a146a admin admin] RBAC: Authorizing `identity:create_grant(project_id=f096e399deac4d029ff63e3cb7f1ce9f, user_id=61bc9ad2c9444fc297eb4b5902a67b98, role_id=6876e57cc51c448097ab109d35b2a3da)` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:38:38.403840 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1445680c-1b66-40cd-a07e-f55d1d2a146a admin admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:38:38.422803 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1308/2607] 10.222.0.49 () {60 vars in 1482 bytes} [Wed Jun 15 10:38:38 2022] PUT /identity/v3/projects/f096e399deac4d029ff63e3cb7f1ce9f/users/61bc9ad2c9444fc297eb4b5902a67b98/roles/6876e57cc51c448097ab109d35b2a3da => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 10:38:38.425219 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e2d7f6-384c-4ea4-b2b8-c89eeac5994f None None] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:38:38.425586 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e2d7f6-384c-4ea4-b2b8-c89eeac5994f None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:38:38.425921 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e2d7f6-384c-4ea4-b2b8-c89eeac5994f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:38:38.447405 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.auth.core [None req-35e2d7f6-384c-4ea4-b2b8-c89eeac5994f None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:38:38.451396 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-35e2d7f6-384c-4ea4-b2b8-c89eeac5994f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:38:38.455777 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-a466f672-e44e-4ac3-a59c-0d68668883f9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b580057e18d34d929df41ef8f21ed67a'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:38:38.462947 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-a466f672-e44e-4ac3-a59c-0d68668883f9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b580057e18d34d929df41ef8f21ed67a'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:38:38.463732 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-a466f672-e44e-4ac3-a59c-0d68668883f9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b580057e18d34d929df41ef8f21ed67a'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:38:38.464358 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-a466f672-e44e-4ac3-a59c-0d68668883f9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b580057e18d34d929df41ef8f21ed67a'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:38:38.471329 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-a466f672-e44e-4ac3-a59c-0d68668883f9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b580057e18d34d929df41ef8f21ed67a'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:38:38.489145 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1300/2608] 10.222.0.49 () {58 vars in 1223 bytes} [Wed Jun 15 10:38:38 2022] DELETE /identity/v3/users/b580057e18d34d929df41ef8f21ed67a => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 10:38:38.514985 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1309/2609] 10.222.0.49 () {58 vars in 957 bytes} [Wed Jun 15 10:38:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:38:38.524872 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a466f672-e44e-4ac3-a59c-0d68668883f9 admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:38:38.526301 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-a466f672-e44e-4ac3-a59c-0d68668883f9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:38:38.599762 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a466f672-e44e-4ac3-a59c-0d68668883f9 admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:38:38.603831 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257484da-1157-4759-8a97-04194f9b734a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0cb44a17d205431793e9a8791e477a32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '559b3e713f5844b6ae37151d56dd419f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:38:38.604627 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257484da-1157-4759-8a97-04194f9b734a service neutron] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:38:38.604937 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257484da-1157-4759-8a97-04194f9b734a service neutron] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:38:38.605273 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257484da-1157-4759-8a97-04194f9b734a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:38:38.676496 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-257484da-1157-4759-8a97-04194f9b734a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:38:38.677942 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-257484da-1157-4759-8a97-04194f9b734a service neutron] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:38:38.689188 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1301/2610] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Jun 15 10:38:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:38:38.831276 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e2d7f6-384c-4ea4-b2b8-c89eeac5994f None None] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:38:38.832747 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-35e2d7f6-384c-4ea4-b2b8-c89eeac5994f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:38:38.856030 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e2d7f6-384c-4ea4-b2b8-c89eeac5994f None None] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:38:38.865077 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3098e9dd-30ae-438d-82f4-465e300ad964 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:38:38.866271 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3098e9dd-30ae-438d-82f4-465e300ad964 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:38:38.866564 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3098e9dd-30ae-438d-82f4-465e300ad964 admin admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:38:38.866923 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3098e9dd-30ae-438d-82f4-465e300ad964 admin admin] PATH_INFO: `/v3/projects/5bde1122d78b42b3a44362c818ea9cae` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:38:38.868369 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3098e9dd-30ae-438d-82f4-465e300ad964 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5bde1122d78b42b3a44362c818ea9cae)` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:38:38.870561 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3098e9dd-30ae-438d-82f4-465e300ad964 admin admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:38:38.907861 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=95,keystone:INSERT=4,keystone:DELETE=1 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:38:38.929710 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-3098e9dd-30ae-438d-82f4-465e300ad964 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5bde1122d78b42b3a44362c818ea9cae', 'request_id': 'req-3098e9dd-30ae-438d-82f4-465e300ad964'} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:38:38.937446 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-3098e9dd-30ae-438d-82f4-465e300ad964 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5bde1122d78b42b3a44362c818ea9cae', 'request_id': 'req-3098e9dd-30ae-438d-82f4-465e300ad964'} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:38:38.947609 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-3098e9dd-30ae-438d-82f4-465e300ad964 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5bde1122d78b42b3a44362c818ea9cae', 'request_id': 'req-3098e9dd-30ae-438d-82f4-465e300ad964'} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:38:38.961731 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-3098e9dd-30ae-438d-82f4-465e300ad964 admin admin] The token cache is being invalidate because project 5bde1122d78b42b3a44362c818ea9cae was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80119) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 10:38:38.962020 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-3098e9dd-30ae-438d-82f4-465e300ad964 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:38:38.962698 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-3098e9dd-30ae-438d-82f4-465e300ad964 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:38:38.964547 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1310/2611] 10.222.0.49 () {58 vars in 1232 bytes} [Wed Jun 15 10:38:38 2022] DELETE /identity/v3/projects/5bde1122d78b42b3a44362c818ea9cae => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 10:38:38.969198 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ed196d-f478-429e-849f-8b34e5609cb4 None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:38:38.970020 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ed196d-f478-429e-849f-8b34e5609cb4 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:38:38.970020 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ed196d-f478-429e-849f-8b34e5609cb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:38:38.994110 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-27ed196d-f478-429e-849f-8b34e5609cb4 None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:38:38.995959 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-27ed196d-f478-429e-849f-8b34e5609cb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:38:39.042537 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:38:39.130657 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1302/2612] 10.222.0.49 () {58 vars in 957 bytes} [Wed Jun 15 10:38:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:38:39.139025 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220b0558-2d8d-4c3c-83aa-239cef948999 None None] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:38:39.139362 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220b0558-2d8d-4c3c-83aa-239cef948999 None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:38:39.139645 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220b0558-2d8d-4c3c-83aa-239cef948999 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:38:39.161310 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.auth.core [None req-220b0558-2d8d-4c3c-83aa-239cef948999 None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:38:39.162587 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-220b0558-2d8d-4c3c-83aa-239cef948999 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:38:39.171382 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1311/2613] 10.222.0.49 () {58 vars in 957 bytes} [Wed Jun 15 10:38:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 10:38:39.175822 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ed196d-f478-429e-849f-8b34e5609cb4 None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:38:39.187169 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ed196d-f478-429e-849f-8b34e5609cb4 None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:38:39.195822 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22ad099-ea95-4b48-9917-d4758d68d3b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:38:39.196632 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22ad099-ea95-4b48-9917-d4758d68d3b8 admin admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:38:39.196967 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22ad099-ea95-4b48-9917-d4758d68d3b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:38:39.199337 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22ad099-ea95-4b48-9917-d4758d68d3b8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:38:39.200458 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22ad099-ea95-4b48-9917-d4758d68d3b8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:38:39.201038 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22ad099-ea95-4b48-9917-d4758d68d3b8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:38:39.206002 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22ad099-ea95-4b48-9917-d4758d68d3b8 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:38:39.222846 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1303/2614] 10.222.0.49 () {58 vars in 1152 bytes} [Wed Jun 15 10:38:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:38:39.227078 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220b0558-2d8d-4c3c-83aa-239cef948999 None None] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:38:39.239980 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220b0558-2d8d-4c3c-83aa-239cef948999 None None] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:38:39.244976 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6dac6e-70dc-48b6-b402-0f6c5245572d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:38:39.245651 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6dac6e-70dc-48b6-b402-0f6c5245572d admin admin] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:38:39.245944 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6dac6e-70dc-48b6-b402-0f6c5245572d admin admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:38:39.246267 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6dac6e-70dc-48b6-b402-0f6c5245572d admin admin] PATH_INFO: `/v3/projects` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:38:39.246975 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d6dac6e-70dc-48b6-b402-0f6c5245572d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:38:39.249076 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d6dac6e-70dc-48b6-b402-0f6c5245572d admin admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:38:39.285673 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1312/2615] 10.222.0.49 () {60 vars in 1152 bytes} [Wed Jun 15 10:38:39 2022] POST /identity/v3/projects => generated 381 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 10:38:39.286970 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22ad099-ea95-4b48-9917-d4758d68d3b8 admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:38:39.298467 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22ad099-ea95-4b48-9917-d4758d68d3b8 admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:38:39.361428 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf8a44d-9529-4119-9874-8b6337c21a2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:38:39.362421 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf8a44d-9529-4119-9874-8b6337c21a2e admin admin] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:38:39.362713 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf8a44d-9529-4119-9874-8b6337c21a2e admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:38:39.362996 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf8a44d-9529-4119-9874-8b6337c21a2e admin admin] PATH_INFO: `/v3/users` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:38:39.363791 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf8a44d-9529-4119-9874-8b6337c21a2e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:38:39.365785 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf8a44d-9529-4119-9874-8b6337c21a2e admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:38:39.387054 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1304/2616] 10.222.0.49 () {60 vars in 1143 bytes} [Wed Jun 15 10:38:39 2022] POST /identity/v3/users => generated 291 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 10:38:39.391069 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6dac6e-70dc-48b6-b402-0f6c5245572d admin admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:38:39.402042 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6dac6e-70dc-48b6-b402-0f6c5245572d admin admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:38:39.408237 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0498ac7-8d50-4698-8cb3-f1386c65ad9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:38:39.408923 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0498ac7-8d50-4698-8cb3-f1386c65ad9c admin admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:38:39.409210 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0498ac7-8d50-4698-8cb3-f1386c65ad9c admin admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:38:39.409493 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0498ac7-8d50-4698-8cb3-f1386c65ad9c admin admin] PATH_INFO: `/v3/roles` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:38:39.410227 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0498ac7-8d50-4698-8cb3-f1386c65ad9c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:38:39.411978 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0498ac7-8d50-4698-8cb3-f1386c65ad9c admin admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:38:39.427219 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1313/2617] 10.222.0.49 () {58 vars in 1121 bytes} [Wed Jun 15 10:38:39 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 10:38:39.429145 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf8a44d-9529-4119-9874-8b6337c21a2e admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:38:39.441999 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf8a44d-9529-4119-9874-8b6337c21a2e admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:38:39.450069 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444569d9-2b0a-4bde-8851-e9b21dd68764 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:38:39.450756 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444569d9-2b0a-4bde-8851-e9b21dd68764 admin admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:38:39.451039 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444569d9-2b0a-4bde-8851-e9b21dd68764 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:38:39.451348 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444569d9-2b0a-4bde-8851-e9b21dd68764 admin admin] PATH_INFO: `/v3/roles` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:38:39.452187 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-444569d9-2b0a-4bde-8851-e9b21dd68764 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:38:39.453787 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-444569d9-2b0a-4bde-8851-e9b21dd68764 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:38:39.465041 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1305/2618] 10.222.0.49 () {58 vars in 1121 bytes} [Wed Jun 15 10:38:39 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 10:38:39.468936 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0498ac7-8d50-4698-8cb3-f1386c65ad9c admin admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:38:39.480045 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0498ac7-8d50-4698-8cb3-f1386c65ad9c admin admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:38:39.487046 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196b1d59-1f0f-4b95-9350-510653ef9d04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:38:39.488312 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196b1d59-1f0f-4b95-9350-510653ef9d04 admin admin] REQUEST_METHOD: `PUT` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:38:39.488598 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196b1d59-1f0f-4b95-9350-510653ef9d04 admin admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:38:39.488873 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196b1d59-1f0f-4b95-9350-510653ef9d04 admin admin] PATH_INFO: `/v3/projects/e95bd36bbe3642169c31720ff0a1aed2/users/f236964553bc46009d1cb9a600da4ec9/roles/6876e57cc51c448097ab109d35b2a3da` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:38:39.488986 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444569d9-2b0a-4bde-8851-e9b21dd68764 admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:38:39.490951 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-444569d9-2b0a-4bde-8851-e9b21dd68764 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:38:39.511820 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444569d9-2b0a-4bde-8851-e9b21dd68764 admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:38:39.513783 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-196b1d59-1f0f-4b95-9350-510653ef9d04 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e95bd36bbe3642169c31720ff0a1aed2, user_id=f236964553bc46009d1cb9a600da4ec9, role_id=6876e57cc51c448097ab109d35b2a3da)` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:38:39.517290 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-196b1d59-1f0f-4b95-9350-510653ef9d04 admin admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:38:39.519212 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa5774e-2b8b-4a6e-841e-e732424c4ef4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:38:39.520326 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa5774e-2b8b-4a6e-841e-e732424c4ef4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:38:39.520781 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa5774e-2b8b-4a6e-841e-e732424c4ef4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:38:39.521257 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa5774e-2b8b-4a6e-841e-e732424c4ef4 admin admin] PATH_INFO: `/v3/users/873b3e33e2b944a7a6daec8e3f55c5d0` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:38:39.522715 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa5774e-2b8b-4a6e-841e-e732424c4ef4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=873b3e33e2b944a7a6daec8e3f55c5d0)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:38:39.525029 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa5774e-2b8b-4a6e-841e-e732424c4ef4 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:38:39.538602 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1314/2619] 10.222.0.49 () {60 vars in 1482 bytes} [Wed Jun 15 10:38:39 2022] PUT /identity/v3/projects/e95bd36bbe3642169c31720ff0a1aed2/users/f236964553bc46009d1cb9a600da4ec9/roles/6876e57cc51c448097ab109d35b2a3da => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:38:39.550321 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aea0cbe-6fc6-4b2d-b777-c58e765974fe None None] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:38:39.550636 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aea0cbe-6fc6-4b2d-b777-c58e765974fe None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:38:39.550895 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aea0cbe-6fc6-4b2d-b777-c58e765974fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:38:39.588038 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-7fa5774e-2b8b-4a6e-841e-e732424c4ef4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '873b3e33e2b944a7a6daec8e3f55c5d0'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:38:39.589894 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.auth.core [None req-3aea0cbe-6fc6-4b2d-b777-c58e765974fe None None] MFA Rules not processed for user `f236964553bc46009d1cb9a600da4ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=80119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:38:39.591514 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-3aea0cbe-6fc6-4b2d-b777-c58e765974fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:38:39.600988 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-7fa5774e-2b8b-4a6e-841e-e732424c4ef4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '873b3e33e2b944a7a6daec8e3f55c5d0'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:38:39.602270 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-7fa5774e-2b8b-4a6e-841e-e732424c4ef4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '873b3e33e2b944a7a6daec8e3f55c5d0'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:38:39.603153 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-7fa5774e-2b8b-4a6e-841e-e732424c4ef4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '873b3e33e2b944a7a6daec8e3f55c5d0'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:38:39.614855 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-7fa5774e-2b8b-4a6e-841e-e732424c4ef4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '873b3e33e2b944a7a6daec8e3f55c5d0'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:38:39.637073 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1306/2620] 10.222.0.49 () {58 vars in 1223 bytes} [Wed Jun 15 10:38:39 2022] DELETE /identity/v3/users/873b3e33e2b944a7a6daec8e3f55c5d0 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 10:38:39.643694 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a03e0c-5721-4697-99e6-3ea73763fc84 None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:38:39.644183 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a03e0c-5721-4697-99e6-3ea73763fc84 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:38:39.646436 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a03e0c-5721-4697-99e6-3ea73763fc84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:38:39.663330 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:38:39.673698 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-56a03e0c-5721-4697-99e6-3ea73763fc84 None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:38:39.676066 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-56a03e0c-5721-4697-99e6-3ea73763fc84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:38:39.697438 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1315/2621] 10.222.0.49 () {58 vars in 957 bytes} [Wed Jun 15 10:38:39 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:38:39.702875 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aea0cbe-6fc6-4b2d-b777-c58e765974fe None None] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:38:39.707427 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-3aea0cbe-6fc6-4b2d-b777-c58e765974fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:38:39.750117 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1307/2622] 10.222.0.49 () {58 vars in 957 bytes} [Wed Jun 15 10:38:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:38:39.758785 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a03e0c-5721-4697-99e6-3ea73763fc84 None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:38:39.764038 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-56a03e0c-5721-4697-99e6-3ea73763fc84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:38:39.770496 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:DELETE=6,keystone:SELECT=91,keystone:INSERT=3 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:38:39.819910 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aea0cbe-6fc6-4b2d-b777-c58e765974fe None None] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:38:39.828285 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a954a9c-9ee1-4b7f-aa30-23e9bc68f11a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:38:39.829021 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a954a9c-9ee1-4b7f-aa30-23e9bc68f11a admin admin] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:38:39.829308 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a954a9c-9ee1-4b7f-aa30-23e9bc68f11a admin admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:38:39.831195 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a954a9c-9ee1-4b7f-aa30-23e9bc68f11a admin admin] PATH_INFO: `/v3/projects` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:38:39.832424 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a954a9c-9ee1-4b7f-aa30-23e9bc68f11a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:38:39.834932 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a954a9c-9ee1-4b7f-aa30-23e9bc68f11a admin admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:38:39.867528 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a03e0c-5721-4697-99e6-3ea73763fc84 None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:38:39.871658 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dee46f5-1a0a-4f49-9eea-6c1e9d64320b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0cb44a17d205431793e9a8791e477a32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '559b3e713f5844b6ae37151d56dd419f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:38:39.872337 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dee46f5-1a0a-4f49-9eea-6c1e9d64320b service neutron] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:38:39.872636 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dee46f5-1a0a-4f49-9eea-6c1e9d64320b service neutron] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:38:39.872946 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dee46f5-1a0a-4f49-9eea-6c1e9d64320b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:38:39.874464 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1316/2623] 10.222.0.49 () {60 vars in 1152 bytes} [Wed Jun 15 10:38:39 2022] POST /identity/v3/projects => generated 381 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 10:38:39.878368 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a954a9c-9ee1-4b7f-aa30-23e9bc68f11a admin admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:38:39.889694 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dee46f5-1a0a-4f49-9eea-6c1e9d64320b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:38:39.895524 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dee46f5-1a0a-4f49-9eea-6c1e9d64320b service neutron] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:38:39.911152 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1308/2624] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Jun 15 10:38:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:38:39.979971 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a954a9c-9ee1-4b7f-aa30-23e9bc68f11a admin admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:38:39.985814 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6ba71d-c42e-4566-aad3-f828208ab898 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:38:39.986800 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6ba71d-c42e-4566-aad3-f828208ab898 admin admin] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:38:39.988154 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6ba71d-c42e-4566-aad3-f828208ab898 admin admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:38:39.988154 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6ba71d-c42e-4566-aad3-f828208ab898 admin admin] PATH_INFO: `/v3/users` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:38:39.988645 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c6ba71d-c42e-4566-aad3-f828208ab898 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:38:39.991121 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c6ba71d-c42e-4566-aad3-f828208ab898 admin admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:38:40.025713 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dee46f5-1a0a-4f49-9eea-6c1e9d64320b service neutron] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:38:40.033645 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1317/2625] 10.222.0.49 () {60 vars in 1143 bytes} [Wed Jun 15 10:38:39 2022] POST /identity/v3/users => generated 297 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 10:38:40.042611 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dee46f5-1a0a-4f49-9eea-6c1e9d64320b service neutron] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:38:40.050898 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e4dfed-70bd-485e-b453-e32539d6b896 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:38:40.051701 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e4dfed-70bd-485e-b453-e32539d6b896 admin admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:38:40.052052 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e4dfed-70bd-485e-b453-e32539d6b896 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:38:40.052424 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e4dfed-70bd-485e-b453-e32539d6b896 admin admin] PATH_INFO: `/v3/roles` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:38:40.053220 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e4dfed-70bd-485e-b453-e32539d6b896 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:38:40.055262 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e4dfed-70bd-485e-b453-e32539d6b896 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:38:40.079631 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1309/2626] 10.222.0.49 () {58 vars in 1121 bytes} [Wed Jun 15 10:38:40 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 10:38:40.081451 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6ba71d-c42e-4566-aad3-f828208ab898 admin admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:38:40.093315 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6ba71d-c42e-4566-aad3-f828208ab898 admin admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:38:40.100762 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca37fef-ea46-48e0-bc09-c7c7ed04864f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:38:40.102884 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca37fef-ea46-48e0-bc09-c7c7ed04864f admin admin] REQUEST_METHOD: `PUT` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:38:40.102884 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca37fef-ea46-48e0-bc09-c7c7ed04864f admin admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:38:40.102884 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca37fef-ea46-48e0-bc09-c7c7ed04864f admin admin] PATH_INFO: `/v3/projects/347fc73d3eb84c4e89bf0cf160bc52af/users/a57b1195231846b7a72dd4c8a6482f53/roles/8051174e7a7947d5b991ebd793c2e543` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:38:40.126356 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca37fef-ea46-48e0-bc09-c7c7ed04864f admin admin] RBAC: Authorizing `identity:create_grant(project_id=347fc73d3eb84c4e89bf0cf160bc52af, user_id=a57b1195231846b7a72dd4c8a6482f53, role_id=8051174e7a7947d5b991ebd793c2e543)` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:38:40.129550 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca37fef-ea46-48e0-bc09-c7c7ed04864f admin admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:38:40.155629 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1318/2627] 10.222.0.49 () {60 vars in 1482 bytes} [Wed Jun 15 10:38:40 2022] PUT /identity/v3/projects/347fc73d3eb84c4e89bf0cf160bc52af/users/a57b1195231846b7a72dd4c8a6482f53/roles/8051174e7a7947d5b991ebd793c2e543 => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:38:40.160265 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e4dfed-70bd-485e-b453-e32539d6b896 admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:38:40.174103 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e4dfed-70bd-485e-b453-e32539d6b896 admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:38:40.290350 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd665b3d-3043-499a-9510-5381b4039dab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:38:40.291921 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd665b3d-3043-499a-9510-5381b4039dab admin admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:38:40.292462 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd665b3d-3043-499a-9510-5381b4039dab admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:38:40.292994 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd665b3d-3043-499a-9510-5381b4039dab admin admin] PATH_INFO: `/v3/roles` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:38:40.294011 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd665b3d-3043-499a-9510-5381b4039dab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:38:40.302401 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd665b3d-3043-499a-9510-5381b4039dab admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:38:40.330977 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1310/2628] 10.222.0.49 () {58 vars in 1121 bytes} [Wed Jun 15 10:38:40 2022] GET /identity/v3/roles => generated 1420 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 10:38:40.332186 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca37fef-ea46-48e0-bc09-c7c7ed04864f admin admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:38:40.349598 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca37fef-ea46-48e0-bc09-c7c7ed04864f admin admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:38:40.354084 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf79236-b2b2-46a2-89a1-954f31a52e22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:38:40.354998 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf79236-b2b2-46a2-89a1-954f31a52e22 admin admin] REQUEST_METHOD: `PUT` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:38:40.355313 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf79236-b2b2-46a2-89a1-954f31a52e22 admin admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:38:40.355591 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf79236-b2b2-46a2-89a1-954f31a52e22 admin admin] PATH_INFO: `/v3/projects/347fc73d3eb84c4e89bf0cf160bc52af/users/a57b1195231846b7a72dd4c8a6482f53/roles/8051174e7a7947d5b991ebd793c2e543` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:38:40.357727 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf79236-b2b2-46a2-89a1-954f31a52e22 admin admin] RBAC: Authorizing `identity:create_grant(project_id=347fc73d3eb84c4e89bf0cf160bc52af, user_id=a57b1195231846b7a72dd4c8a6482f53, role_id=8051174e7a7947d5b991ebd793c2e543)` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:38:40.360519 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf79236-b2b2-46a2-89a1-954f31a52e22 admin admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:38:40.372557 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1319/2629] 10.222.0.49 () {60 vars in 1482 bytes} [Wed Jun 15 10:38:40 2022] PUT /identity/v3/projects/347fc73d3eb84c4e89bf0cf160bc52af/users/a57b1195231846b7a72dd4c8a6482f53/roles/8051174e7a7947d5b991ebd793c2e543 => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:38:40.384160 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd665b3d-3043-499a-9510-5381b4039dab admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:38:40.388486 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c8d5fb-627d-4743-857d-b7e1b0e7835f None None] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:38:40.388786 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c8d5fb-627d-4743-857d-b7e1b0e7835f None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:38:40.388983 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-fd665b3d-3043-499a-9510-5381b4039dab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:38:40.389434 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c8d5fb-627d-4743-857d-b7e1b0e7835f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:38:40.423033 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd665b3d-3043-499a-9510-5381b4039dab admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:38:40.445945 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.auth.core [None req-d3c8d5fb-627d-4743-857d-b7e1b0e7835f None None] MFA Rules not processed for user `a57b1195231846b7a72dd4c8a6482f53`. Rule list: `[]` (Enabled: `True`). {{(pid=80119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:38:40.447312 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-d3c8d5fb-627d-4743-857d-b7e1b0e7835f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:38:40.491039 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:38:40.551155 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6f8c69-c8c7-460e-a3b9-fb4dcfc33fb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:38:40.552503 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6f8c69-c8c7-460e-a3b9-fb4dcfc33fb9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:38:40.552793 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6f8c69-c8c7-460e-a3b9-fb4dcfc33fb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:38:40.553099 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6f8c69-c8c7-460e-a3b9-fb4dcfc33fb9 admin admin] PATH_INFO: `/v3/projects/7efe1e4c86ad424aaad415f1663d4619` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:38:40.558896 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6f8c69-c8c7-460e-a3b9-fb4dcfc33fb9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7efe1e4c86ad424aaad415f1663d4619)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:38:40.561195 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6f8c69-c8c7-460e-a3b9-fb4dcfc33fb9 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:38:40.585330 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1320/2630] 10.222.0.49 () {58 vars in 957 bytes} [Wed Jun 15 10:38:40 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:38:40.599451 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c8d5fb-627d-4743-857d-b7e1b0e7835f None None] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:38:40.602131 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-d3c8d5fb-627d-4743-857d-b7e1b0e7835f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:38:40.610333 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:38:40.670044 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-8c6f8c69-c8c7-460e-a3b9-fb4dcfc33fb9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7efe1e4c86ad424aaad415f1663d4619', 'request_id': 'req-8c6f8c69-c8c7-460e-a3b9-fb4dcfc33fb9'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:38:40.685647 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-8c6f8c69-c8c7-460e-a3b9-fb4dcfc33fb9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7efe1e4c86ad424aaad415f1663d4619', 'request_id': 'req-8c6f8c69-c8c7-460e-a3b9-fb4dcfc33fb9'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:38:40.704765 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-8c6f8c69-c8c7-460e-a3b9-fb4dcfc33fb9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7efe1e4c86ad424aaad415f1663d4619', 'request_id': 'req-8c6f8c69-c8c7-460e-a3b9-fb4dcfc33fb9'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:38:40.725830 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-8c6f8c69-c8c7-460e-a3b9-fb4dcfc33fb9 admin admin] The token cache is being invalidate because project 7efe1e4c86ad424aaad415f1663d4619 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80120) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 10:38:40.726156 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-8c6f8c69-c8c7-460e-a3b9-fb4dcfc33fb9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:38:40.726852 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-8c6f8c69-c8c7-460e-a3b9-fb4dcfc33fb9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:38:40.732793 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1311/2631] 10.222.0.49 () {58 vars in 1232 bytes} [Wed Jun 15 10:38:40 2022] DELETE /identity/v3/projects/7efe1e4c86ad424aaad415f1663d4619 => generated 0 bytes in 345 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:38:40.734240 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6f8c69-c8c7-460e-a3b9-fb4dcfc33fb9 admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:38:40.737105 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-8c6f8c69-c8c7-460e-a3b9-fb4dcfc33fb9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:38:40.738354 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c8d5fb-627d-4743-857d-b7e1b0e7835f None None] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:38:40.853394 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d48d3e8-de52-4537-8e09-ca6e0163b6ca service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20d1327553404b80aecee87c78fcb52b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '559b3e713f5844b6ae37151d56dd419f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:38:40.854445 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d48d3e8-de52-4537-8e09-ca6e0163b6ca service cinder] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:38:40.854554 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d48d3e8-de52-4537-8e09-ca6e0163b6ca service cinder] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:38:40.854784 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d48d3e8-de52-4537-8e09-ca6e0163b6ca service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:38:40.860634 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-8d48d3e8-de52-4537-8e09-ca6e0163b6ca service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:38:40.902669 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6f8c69-c8c7-460e-a3b9-fb4dcfc33fb9 admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:38:40.908708 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf5efe5-f795-4635-8626-0b6c16d9b6e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:38:40.909311 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf5efe5-f795-4635-8626-0b6c16d9b6e1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:38:40.909609 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf5efe5-f795-4635-8626-0b6c16d9b6e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:38:40.910031 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf5efe5-f795-4635-8626-0b6c16d9b6e1 admin admin] PATH_INFO: `/v3/users/af006d3f6cbd4746b597ea93ab0d21b1` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:38:40.911350 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf5efe5-f795-4635-8626-0b6c16d9b6e1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=af006d3f6cbd4746b597ea93ab0d21b1)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:38:40.914462 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf5efe5-f795-4635-8626-0b6c16d9b6e1 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:38:40.980038 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d48d3e8-de52-4537-8e09-ca6e0163b6ca service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:38:40.987633 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-bcf5efe5-f795-4635-8626-0b6c16d9b6e1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'af006d3f6cbd4746b597ea93ab0d21b1'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:38:40.992165 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d48d3e8-de52-4537-8e09-ca6e0163b6ca service cinder] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:38:40.996072 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-bcf5efe5-f795-4635-8626-0b6c16d9b6e1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'af006d3f6cbd4746b597ea93ab0d21b1'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:38:40.997596 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-bcf5efe5-f795-4635-8626-0b6c16d9b6e1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'af006d3f6cbd4746b597ea93ab0d21b1'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:38:40.998294 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-bcf5efe5-f795-4635-8626-0b6c16d9b6e1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'af006d3f6cbd4746b597ea93ab0d21b1'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:38:41.007299 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-bcf5efe5-f795-4635-8626-0b6c16d9b6e1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'af006d3f6cbd4746b597ea93ab0d21b1'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:38:41.008397 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1321/2632] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Jun 15 10:38:40 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 410 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 10:38:41.009345 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d48d3e8-de52-4537-8e09-ca6e0163b6ca service cinder] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:38:41.010741 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-8d48d3e8-de52-4537-8e09-ca6e0163b6ca service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:38:41.017850 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:38:41.025438 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d48d3e8-de52-4537-8e09-ca6e0163b6ca service cinder] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:38:41.031710 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26efa77-dcf2-4682-abc6-8c8897aabce1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:38:41.032683 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26efa77-dcf2-4682-abc6-8c8897aabce1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:38:41.033033 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26efa77-dcf2-4682-abc6-8c8897aabce1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:38:41.033378 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26efa77-dcf2-4682-abc6-8c8897aabce1 admin admin] PATH_INFO: `/v3/users/1366259de7164616a1da93ddb176e51c` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:38:41.035365 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e26efa77-dcf2-4682-abc6-8c8897aabce1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1366259de7164616a1da93ddb176e51c)` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:38:41.037877 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e26efa77-dcf2-4682-abc6-8c8897aabce1 admin admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:38:41.043025 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1312/2633] 10.222.0.49 () {58 vars in 1223 bytes} [Wed Jun 15 10:38:40 2022] DELETE /identity/v3/users/af006d3f6cbd4746b597ea93ab0d21b1 => generated 0 bytes in 306 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:38:41.088913 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf5efe5-f795-4635-8626-0b6c16d9b6e1 admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:38:41.090960 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-bcf5efe5-f795-4635-8626-0b6c16d9b6e1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:38:41.114712 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-e26efa77-dcf2-4682-abc6-8c8897aabce1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1366259de7164616a1da93ddb176e51c'} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:38:41.123782 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-e26efa77-dcf2-4682-abc6-8c8897aabce1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1366259de7164616a1da93ddb176e51c'} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:38:41.124526 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-e26efa77-dcf2-4682-abc6-8c8897aabce1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1366259de7164616a1da93ddb176e51c'} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:38:41.125231 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-e26efa77-dcf2-4682-abc6-8c8897aabce1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1366259de7164616a1da93ddb176e51c'} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:38:41.133712 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-e26efa77-dcf2-4682-abc6-8c8897aabce1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1366259de7164616a1da93ddb176e51c'} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:38:41.136366 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:38:41.159135 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1322/2634] 10.222.0.49 () {58 vars in 1223 bytes} [Wed Jun 15 10:38:41 2022] DELETE /identity/v3/users/1366259de7164616a1da93ddb176e51c => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:38:41.163505 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d005640a-41ab-499d-b6d4-e1f4da93b878 None None] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:38:41.164601 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d005640a-41ab-499d-b6d4-e1f4da93b878 None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:38:41.164990 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d005640a-41ab-499d-b6d4-e1f4da93b878 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:38:41.171853 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf5efe5-f795-4635-8626-0b6c16d9b6e1 admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:38:41.188165 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.auth.core [None req-d005640a-41ab-499d-b6d4-e1f4da93b878 None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:38:41.189536 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-d005640a-41ab-499d-b6d4-e1f4da93b878 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:38:41.238522 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b767d8b9-eb94-47a1-9cdd-e22cfe57d7d9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20d1327553404b80aecee87c78fcb52b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '559b3e713f5844b6ae37151d56dd419f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:38:41.239554 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b767d8b9-eb94-47a1-9cdd-e22cfe57d7d9 service cinder] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:38:41.239956 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b767d8b9-eb94-47a1-9cdd-e22cfe57d7d9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:38:41.240321 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b767d8b9-eb94-47a1-9cdd-e22cfe57d7d9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:38:41.242849 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-b767d8b9-eb94-47a1-9cdd-e22cfe57d7d9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:38:41.272304 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1323/2635] 10.222.0.49 () {58 vars in 957 bytes} [Wed Jun 15 10:38:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:38:41.278261 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d005640a-41ab-499d-b6d4-e1f4da93b878 None None] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:38:41.282618 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-d005640a-41ab-499d-b6d4-e1f4da93b878 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:38:41.328110 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b767d8b9-eb94-47a1-9cdd-e22cfe57d7d9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:38:41.330122 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b767d8b9-eb94-47a1-9cdd-e22cfe57d7d9 service cinder] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:38:41.349829 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1313/2636] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Jun 15 10:38:41 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 258 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:38:41.406084 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d005640a-41ab-499d-b6d4-e1f4da93b878 None None] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:38:41.411200 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7779ee-4892-477b-87d6-b349f87cd94e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0cb44a17d205431793e9a8791e477a32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '559b3e713f5844b6ae37151d56dd419f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:38:41.411874 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7779ee-4892-477b-87d6-b349f87cd94e service neutron] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:38:41.412162 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7779ee-4892-477b-87d6-b349f87cd94e service neutron] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:38:41.412443 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7779ee-4892-477b-87d6-b349f87cd94e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:38:41.429791 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a7779ee-4892-477b-87d6-b349f87cd94e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:38:41.431588 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a7779ee-4892-477b-87d6-b349f87cd94e service neutron] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:38:41.447272 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1324/2637] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Jun 15 10:38:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 10:38:41.592122 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b767d8b9-eb94-47a1-9cdd-e22cfe57d7d9 service cinder] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:38:41.593816 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-b767d8b9-eb94-47a1-9cdd-e22cfe57d7d9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:38:41.624180 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b767d8b9-eb94-47a1-9cdd-e22cfe57d7d9 service cinder] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:38:41.644295 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e348a145-aefb-41b6-89aa-440c43f2e594 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:38:41.645204 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e348a145-aefb-41b6-89aa-440c43f2e594 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:38:41.645486 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e348a145-aefb-41b6-89aa-440c43f2e594 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:38:41.645763 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e348a145-aefb-41b6-89aa-440c43f2e594 admin admin] PATH_INFO: `/v3/projects/090490cf141f45d9a00afe05675fed43` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:38:41.651238 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e348a145-aefb-41b6-89aa-440c43f2e594 admin admin] RBAC: Authorizing `identity:delete_project(project_id=090490cf141f45d9a00afe05675fed43)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:38:41.653759 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e348a145-aefb-41b6-89aa-440c43f2e594 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:38:41.778893 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-e348a145-aefb-41b6-89aa-440c43f2e594 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '090490cf141f45d9a00afe05675fed43', 'request_id': 'req-e348a145-aefb-41b6-89aa-440c43f2e594'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:38:41.782469 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:DELETE=6,keystone:SELECT=94 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:38:41.798069 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-e348a145-aefb-41b6-89aa-440c43f2e594 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '090490cf141f45d9a00afe05675fed43', 'request_id': 'req-e348a145-aefb-41b6-89aa-440c43f2e594'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:38:41.806428 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-e348a145-aefb-41b6-89aa-440c43f2e594 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '090490cf141f45d9a00afe05675fed43', 'request_id': 'req-e348a145-aefb-41b6-89aa-440c43f2e594'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:38:41.831682 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-e348a145-aefb-41b6-89aa-440c43f2e594 admin admin] The token cache is being invalidate because project 090490cf141f45d9a00afe05675fed43 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80120) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 10:38:41.832632 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-e348a145-aefb-41b6-89aa-440c43f2e594 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:38:41.833470 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-e348a145-aefb-41b6-89aa-440c43f2e594 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:38:41.839579 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1314/2638] 10.222.0.49 () {58 vars in 1232 bytes} [Wed Jun 15 10:38:41 2022] DELETE /identity/v3/projects/090490cf141f45d9a00afe05675fed43 => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:38:41.844859 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754cad95-d875-4fcf-82cd-094182703a12 None None] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:38:41.845162 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754cad95-d875-4fcf-82cd-094182703a12 None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:38:41.845444 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754cad95-d875-4fcf-82cd-094182703a12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:38:41.867642 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e348a145-aefb-41b6-89aa-440c43f2e594 admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:38:41.872183 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-e348a145-aefb-41b6-89aa-440c43f2e594 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:38:41.883235 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.auth.core [None req-754cad95-d875-4fcf-82cd-094182703a12 None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:38:41.884739 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-754cad95-d875-4fcf-82cd-094182703a12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:38:41.993104 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1325/2639] 10.222.0.49 () {58 vars in 957 bytes} [Wed Jun 15 10:38:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 10:38:41.993104 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754cad95-d875-4fcf-82cd-094182703a12 None None] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:38:41.993104 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-754cad95-d875-4fcf-82cd-094182703a12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:38:42.003182 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754cad95-d875-4fcf-82cd-094182703a12 None None] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:38:42.007863 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e348a145-aefb-41b6-89aa-440c43f2e594 admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:38:42.008969 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa655aa-31b8-4fce-9e6b-e1de237918c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:38:42.009926 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa655aa-31b8-4fce-9e6b-e1de237918c0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:38:42.010584 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa655aa-31b8-4fce-9e6b-e1de237918c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:38:42.010943 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa655aa-31b8-4fce-9e6b-e1de237918c0 admin admin] PATH_INFO: `/v3/projects/4fc87baa62314a1ca6fd85e95e3b5782` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:38:42.012264 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa655aa-31b8-4fce-9e6b-e1de237918c0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4fc87baa62314a1ca6fd85e95e3b5782)` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:38:42.013033 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9d5047-4044-430e-8487-19cc3f7f7b09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:38:42.013921 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9d5047-4044-430e-8487-19cc3f7f7b09 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:38:42.014209 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9d5047-4044-430e-8487-19cc3f7f7b09 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:38:42.014493 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9d5047-4044-430e-8487-19cc3f7f7b09 admin admin] PATH_INFO: `/v3/users/f236964553bc46009d1cb9a600da4ec9` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:38:42.014675 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa655aa-31b8-4fce-9e6b-e1de237918c0 admin admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:38:42.017224 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d9d5047-4044-430e-8487-19cc3f7f7b09 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f236964553bc46009d1cb9a600da4ec9)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:38:42.019888 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d9d5047-4044-430e-8487-19cc3f7f7b09 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:38:42.044797 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:38:42.080414 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-faa655aa-31b8-4fce-9e6b-e1de237918c0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4fc87baa62314a1ca6fd85e95e3b5782', 'request_id': 'req-faa655aa-31b8-4fce-9e6b-e1de237918c0'} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:38:42.086036 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-3d9d5047-4044-430e-8487-19cc3f7f7b09 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f236964553bc46009d1cb9a600da4ec9'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:38:42.089646 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-faa655aa-31b8-4fce-9e6b-e1de237918c0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4fc87baa62314a1ca6fd85e95e3b5782', 'request_id': 'req-faa655aa-31b8-4fce-9e6b-e1de237918c0'} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:38:42.100100 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-3d9d5047-4044-430e-8487-19cc3f7f7b09 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f236964553bc46009d1cb9a600da4ec9'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:38:42.101027 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-3d9d5047-4044-430e-8487-19cc3f7f7b09 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f236964553bc46009d1cb9a600da4ec9'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:38:42.101889 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-3d9d5047-4044-430e-8487-19cc3f7f7b09 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f236964553bc46009d1cb9a600da4ec9'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:38:42.102309 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-faa655aa-31b8-4fce-9e6b-e1de237918c0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4fc87baa62314a1ca6fd85e95e3b5782', 'request_id': 'req-faa655aa-31b8-4fce-9e6b-e1de237918c0'} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:38:42.112532 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-3d9d5047-4044-430e-8487-19cc3f7f7b09 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f236964553bc46009d1cb9a600da4ec9'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:38:42.115823 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-faa655aa-31b8-4fce-9e6b-e1de237918c0 admin admin] The token cache is being invalidate because project 4fc87baa62314a1ca6fd85e95e3b5782 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80119) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 10:38:42.116350 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-faa655aa-31b8-4fce-9e6b-e1de237918c0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:38:42.117867 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-faa655aa-31b8-4fce-9e6b-e1de237918c0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:38:42.120106 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1326/2640] 10.222.0.49 () {58 vars in 1232 bytes} [Wed Jun 15 10:38:41 2022] DELETE /identity/v3/projects/4fc87baa62314a1ca6fd85e95e3b5782 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:38:42.121611 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc5802f-c55b-4dc9-a539-272021ff2c9b None None] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:38:42.122214 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc5802f-c55b-4dc9-a539-272021ff2c9b None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:38:42.123648 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc5802f-c55b-4dc9-a539-272021ff2c9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:38:42.152035 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.auth.core [None req-6bc5802f-c55b-4dc9-a539-272021ff2c9b None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:38:42.154026 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-6bc5802f-c55b-4dc9-a539-272021ff2c9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:38:42.157211 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1315/2641] 10.222.0.49 () {58 vars in 1223 bytes} [Wed Jun 15 10:38:41 2022] DELETE /identity/v3/users/f236964553bc46009d1cb9a600da4ec9 => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:38:42.158957 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9d5047-4044-430e-8487-19cc3f7f7b09 admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:38:42.163479 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-3d9d5047-4044-430e-8487-19cc3f7f7b09 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:38:42.249063 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1327/2642] 10.222.0.49 () {58 vars in 957 bytes} [Wed Jun 15 10:38:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:38:42.251997 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12228b59-6ffb-41e7-8f8d-174276002cb0 None None] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:38:42.252313 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12228b59-6ffb-41e7-8f8d-174276002cb0 None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:38:42.253389 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12228b59-6ffb-41e7-8f8d-174276002cb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:38:42.253842 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:INSERT=2,keystone:DELETE=13,keystone:SELECT=85 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:38:42.261101 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9d5047-4044-430e-8487-19cc3f7f7b09 admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:38:42.269588 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5a8a39-6803-4ef2-bd61-916ae4d45c23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:38:42.270781 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5a8a39-6803-4ef2-bd61-916ae4d45c23 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:38:42.271193 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5a8a39-6803-4ef2-bd61-916ae4d45c23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:38:42.271835 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5a8a39-6803-4ef2-bd61-916ae4d45c23 admin admin] PATH_INFO: `/v3/users/5b939993075c442192cad4befc704b00` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:38:42.273941 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b5a8a39-6803-4ef2-bd61-916ae4d45c23 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5b939993075c442192cad4befc704b00)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:38:42.276690 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b5a8a39-6803-4ef2-bd61-916ae4d45c23 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:38:42.279234 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.auth.core [None req-12228b59-6ffb-41e7-8f8d-174276002cb0 None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:38:42.282124 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-12228b59-6ffb-41e7-8f8d-174276002cb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:38:42.299047 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1328/2643] 10.222.0.49 () {58 vars in 957 bytes} [Wed Jun 15 10:38:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:38:42.300166 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12228b59-6ffb-41e7-8f8d-174276002cb0 None None] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:38:42.315531 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12228b59-6ffb-41e7-8f8d-174276002cb0 None None] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:38:42.324056 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d4351d-fd4e-42a3-a108-db8535b94405 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:38:42.324707 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d4351d-fd4e-42a3-a108-db8535b94405 admin admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:38:42.324941 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d4351d-fd4e-42a3-a108-db8535b94405 admin admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:38:42.325225 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d4351d-fd4e-42a3-a108-db8535b94405 admin admin] PATH_INFO: `/v3/domains` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:38:42.326271 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d4351d-fd4e-42a3-a108-db8535b94405 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:38:42.326524 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d4351d-fd4e-42a3-a108-db8535b94405 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:38:42.329295 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d4351d-fd4e-42a3-a108-db8535b94405 admin admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:38:42.347259 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1329/2644] 10.222.0.49 () {58 vars in 1152 bytes} [Wed Jun 15 10:38:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:38:42.348181 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d4351d-fd4e-42a3-a108-db8535b94405 admin admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:38:42.350241 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-e1d4351d-fd4e-42a3-a108-db8535b94405 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:38:42.355023 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-0b5a8a39-6803-4ef2-bd61-916ae4d45c23 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5b939993075c442192cad4befc704b00'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:38:42.374615 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-0b5a8a39-6803-4ef2-bd61-916ae4d45c23 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5b939993075c442192cad4befc704b00'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:38:42.375307 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-0b5a8a39-6803-4ef2-bd61-916ae4d45c23 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5b939993075c442192cad4befc704b00'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:38:42.376631 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-0b5a8a39-6803-4ef2-bd61-916ae4d45c23 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5b939993075c442192cad4befc704b00'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:38:42.391856 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-0b5a8a39-6803-4ef2-bd61-916ae4d45c23 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5b939993075c442192cad4befc704b00'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:38:42.426381 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1316/2645] 10.222.0.49 () {58 vars in 1223 bytes} [Wed Jun 15 10:38:42 2022] DELETE /identity/v3/users/5b939993075c442192cad4befc704b00 => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:38:42.427073 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5a8a39-6803-4ef2-bd61-916ae4d45c23 admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:38:42.428465 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-0b5a8a39-6803-4ef2-bd61-916ae4d45c23 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:38:42.449343 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d4351d-fd4e-42a3-a108-db8535b94405 admin admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:38:42.531683 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:38:42.538695 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5a8a39-6803-4ef2-bd61-916ae4d45c23 admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:38:42.545132 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36a3009-9692-4367-a414-d415ee3efe37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:38:42.546022 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36a3009-9692-4367-a414-d415ee3efe37 admin admin] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:38:42.546022 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36a3009-9692-4367-a414-d415ee3efe37 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:38:42.546216 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36a3009-9692-4367-a414-d415ee3efe37 admin admin] PATH_INFO: `/v3/projects` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:38:42.546849 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d36a3009-9692-4367-a414-d415ee3efe37 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:38:42.549656 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d36a3009-9692-4367-a414-d415ee3efe37 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:38:42.552589 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5f65ac-02e5-46da-8ae9-695d068b4e5b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0cb44a17d205431793e9a8791e477a32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '559b3e713f5844b6ae37151d56dd419f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:38:42.553304 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5f65ac-02e5-46da-8ae9-695d068b4e5b service neutron] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:38:42.555409 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5f65ac-02e5-46da-8ae9-695d068b4e5b service neutron] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:38:42.555732 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5f65ac-02e5-46da-8ae9-695d068b4e5b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:38:42.560870 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-1c5f65ac-02e5-46da-8ae9-695d068b4e5b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:38:42.588011 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1317/2646] 10.222.0.49 () {60 vars in 1152 bytes} [Wed Jun 15 10:38:42 2022] POST /identity/v3/projects => generated 385 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 10:38:42.592674 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36a3009-9692-4367-a414-d415ee3efe37 admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:38:42.593396 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5f65ac-02e5-46da-8ae9-695d068b4e5b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:38:42.595277 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5f65ac-02e5-46da-8ae9-695d068b4e5b service neutron] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:38:42.613151 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1330/2647] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Jun 15 10:38:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 265 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:38:42.689855 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36a3009-9692-4367-a414-d415ee3efe37 admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:38:42.694727 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9139e7-e50a-416f-9875-88746a01b19c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:38:42.695443 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9139e7-e50a-416f-9875-88746a01b19c admin admin] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:38:42.695736 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9139e7-e50a-416f-9875-88746a01b19c admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:38:42.696021 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9139e7-e50a-416f-9875-88746a01b19c admin admin] PATH_INFO: `/v3/users` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:38:42.696731 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b9139e7-e50a-416f-9875-88746a01b19c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:38:42.698670 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b9139e7-e50a-416f-9875-88746a01b19c admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:38:42.711405 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=87,keystone:DELETE=10,keystone:INSERT=3 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:38:42.725136 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1318/2648] 10.222.0.49 () {60 vars in 1143 bytes} [Wed Jun 15 10:38:42 2022] POST /identity/v3/users => generated 293 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 10:38:42.729419 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5f65ac-02e5-46da-8ae9-695d068b4e5b service neutron] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:38:42.739832 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5f65ac-02e5-46da-8ae9-695d068b4e5b service neutron] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:38:42.745829 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e73c14-8e31-4455-9566-0f67e00933e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:38:42.746588 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e73c14-8e31-4455-9566-0f67e00933e6 admin admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:38:42.747282 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e73c14-8e31-4455-9566-0f67e00933e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:38:42.747282 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e73c14-8e31-4455-9566-0f67e00933e6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:38:42.748122 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e73c14-8e31-4455-9566-0f67e00933e6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:38:42.749854 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e73c14-8e31-4455-9566-0f67e00933e6 admin admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:38:42.768460 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1331/2649] 10.222.0.49 () {58 vars in 1121 bytes} [Wed Jun 15 10:38:42 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 10:38:42.772796 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9139e7-e50a-416f-9875-88746a01b19c admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:38:42.796390 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9139e7-e50a-416f-9875-88746a01b19c admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:38:42.801225 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c31072-266f-46ec-9ea3-cb1d84639a0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:38:42.802098 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c31072-266f-46ec-9ea3-cb1d84639a0e admin admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:38:42.802452 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c31072-266f-46ec-9ea3-cb1d84639a0e admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:38:42.802799 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c31072-266f-46ec-9ea3-cb1d84639a0e admin admin] PATH_INFO: `/v3/roles` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:38:42.803638 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c31072-266f-46ec-9ea3-cb1d84639a0e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:38:42.805314 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c31072-266f-46ec-9ea3-cb1d84639a0e admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:38:42.817411 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1319/2650] 10.222.0.49 () {58 vars in 1121 bytes} [Wed Jun 15 10:38:42 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 10:38:42.821512 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e73c14-8e31-4455-9566-0f67e00933e6 admin admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:38:42.832908 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e73c14-8e31-4455-9566-0f67e00933e6 admin admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:38:42.838566 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade0fa1e-ddf2-4e1f-84e4-017d0d89d5a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:38:42.839603 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade0fa1e-ddf2-4e1f-84e4-017d0d89d5a7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:38:42.839940 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade0fa1e-ddf2-4e1f-84e4-017d0d89d5a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:38:42.840285 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade0fa1e-ddf2-4e1f-84e4-017d0d89d5a7 admin admin] PATH_INFO: `/v3/projects/25994ed2792548f099025cfe68c94d97/users/c647d6ce46574dc987a4be6ff4a8d7f7/roles/6876e57cc51c448097ab109d35b2a3da` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:38:42.859767 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade0fa1e-ddf2-4e1f-84e4-017d0d89d5a7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=25994ed2792548f099025cfe68c94d97, user_id=c647d6ce46574dc987a4be6ff4a8d7f7, role_id=6876e57cc51c448097ab109d35b2a3da)` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:38:42.862006 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade0fa1e-ddf2-4e1f-84e4-017d0d89d5a7 admin admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:38:42.877372 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1332/2651] 10.222.0.49 () {60 vars in 1482 bytes} [Wed Jun 15 10:38:42 2022] PUT /identity/v3/projects/25994ed2792548f099025cfe68c94d97/users/c647d6ce46574dc987a4be6ff4a8d7f7/roles/6876e57cc51c448097ab109d35b2a3da => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:38:42.888539 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c896605-177c-4ac6-a0b6-2debe9e06c7c None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:38:42.888972 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c896605-177c-4ac6-a0b6-2debe9e06c7c None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:38:42.889357 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c896605-177c-4ac6-a0b6-2debe9e06c7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:38:42.924197 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-5c896605-177c-4ac6-a0b6-2debe9e06c7c None None] MFA Rules not processed for user `c647d6ce46574dc987a4be6ff4a8d7f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:38:42.940323 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-5c896605-177c-4ac6-a0b6-2debe9e06c7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:38:43.062126 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1320/2652] 10.222.0.49 () {58 vars in 957 bytes} [Wed Jun 15 10:38:42 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:38:43.064032 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade0fa1e-ddf2-4e1f-84e4-017d0d89d5a7 admin admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:38:43.066609 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-ade0fa1e-ddf2-4e1f-84e4-017d0d89d5a7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:38:43.075731 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c896605-177c-4ac6-a0b6-2debe9e06c7c None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:38:43.082628 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-5c896605-177c-4ac6-a0b6-2debe9e06c7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:38:43.182583 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c896605-177c-4ac6-a0b6-2debe9e06c7c None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:38:43.187239 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3085d0-331f-434e-9e6c-5e37d9e39484 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20d1327553404b80aecee87c78fcb52b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '559b3e713f5844b6ae37151d56dd419f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:38:43.188038 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3085d0-331f-434e-9e6c-5e37d9e39484 service cinder] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:38:43.188394 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3085d0-331f-434e-9e6c-5e37d9e39484 service cinder] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:38:43.189011 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3085d0-331f-434e-9e6c-5e37d9e39484 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:38:43.196869 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade0fa1e-ddf2-4e1f-84e4-017d0d89d5a7 admin admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:38:43.200237 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b3085d0-331f-434e-9e6c-5e37d9e39484 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:38:43.201534 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b3085d0-331f-434e-9e6c-5e37d9e39484 service cinder] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:38:43.204570 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed8e266-fce1-4a51-9b62-8df8f834bbb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:38:43.207424 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed8e266-fce1-4a51-9b62-8df8f834bbb2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:38:43.209217 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed8e266-fce1-4a51-9b62-8df8f834bbb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:38:43.210222 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed8e266-fce1-4a51-9b62-8df8f834bbb2 admin admin] PATH_INFO: `/v3/projects/30ed0e4bace545d797e93ac15b5ad45a` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:38:43.212576 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed8e266-fce1-4a51-9b62-8df8f834bbb2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=30ed0e4bace545d797e93ac15b5ad45a)` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:38:43.213773 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1321/2653] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Jun 15 10:38:43 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:38:43.215882 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed8e266-fce1-4a51-9b62-8df8f834bbb2 admin admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:38:43.242070 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3085d0-331f-434e-9e6c-5e37d9e39484 service cinder] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:38:43.255698 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3085d0-331f-434e-9e6c-5e37d9e39484 service cinder] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:38:43.261786 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca416fa-084d-4690-a232-12278178043a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:38:43.262849 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca416fa-084d-4690-a232-12278178043a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:38:43.263255 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca416fa-084d-4690-a232-12278178043a admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:38:43.263835 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca416fa-084d-4690-a232-12278178043a admin admin] PATH_INFO: `/v3/users/c647d6ce46574dc987a4be6ff4a8d7f7` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:38:43.265875 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca416fa-084d-4690-a232-12278178043a admin admin] RBAC: Authorizing `identity:delete_user(user_id=c647d6ce46574dc987a4be6ff4a8d7f7)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:38:43.269239 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca416fa-084d-4690-a232-12278178043a admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:38:43.303186 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-5ed8e266-fce1-4a51-9b62-8df8f834bbb2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '30ed0e4bace545d797e93ac15b5ad45a', 'request_id': 'req-5ed8e266-fce1-4a51-9b62-8df8f834bbb2'} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:38:43.333068 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-5ed8e266-fce1-4a51-9b62-8df8f834bbb2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '30ed0e4bace545d797e93ac15b5ad45a', 'request_id': 'req-5ed8e266-fce1-4a51-9b62-8df8f834bbb2'} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:38:43.348183 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-5ed8e266-fce1-4a51-9b62-8df8f834bbb2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '30ed0e4bace545d797e93ac15b5ad45a', 'request_id': 'req-5ed8e266-fce1-4a51-9b62-8df8f834bbb2'} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:38:43.360762 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-eca416fa-084d-4690-a232-12278178043a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c647d6ce46574dc987a4be6ff4a8d7f7'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:38:43.371169 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-5ed8e266-fce1-4a51-9b62-8df8f834bbb2 admin admin] The token cache is being invalidate because project 30ed0e4bace545d797e93ac15b5ad45a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80119) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 10:38:43.371476 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-5ed8e266-fce1-4a51-9b62-8df8f834bbb2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:38:43.372178 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-5ed8e266-fce1-4a51-9b62-8df8f834bbb2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:38:43.374702 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1333/2654] 10.222.0.49 () {58 vars in 1232 bytes} [Wed Jun 15 10:38:43 2022] DELETE /identity/v3/projects/30ed0e4bace545d797e93ac15b5ad45a => generated 0 bytes in 311 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:38:43.376240 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed8e266-fce1-4a51-9b62-8df8f834bbb2 admin admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:38:43.377604 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-5ed8e266-fce1-4a51-9b62-8df8f834bbb2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:38:43.381338 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-eca416fa-084d-4690-a232-12278178043a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c647d6ce46574dc987a4be6ff4a8d7f7'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:38:43.382245 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-eca416fa-084d-4690-a232-12278178043a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c647d6ce46574dc987a4be6ff4a8d7f7'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:38:43.383065 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-eca416fa-084d-4690-a232-12278178043a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c647d6ce46574dc987a4be6ff4a8d7f7'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:38:43.396370 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-eca416fa-084d-4690-a232-12278178043a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c647d6ce46574dc987a4be6ff4a8d7f7'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:38:43.409079 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:INSERT=3,keystone:SELECT=89,keystone:DELETE=8 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:38:43.440822 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:38:43.466205 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed8e266-fce1-4a51-9b62-8df8f834bbb2 admin admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:38:43.472816 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6eb0143-cc9e-42f4-8b85-1c16cdccd21a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:38:43.474424 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6eb0143-cc9e-42f4-8b85-1c16cdccd21a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:38:43.474720 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6eb0143-cc9e-42f4-8b85-1c16cdccd21a admin admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:38:43.475002 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6eb0143-cc9e-42f4-8b85-1c16cdccd21a admin admin] PATH_INFO: `/v3/projects/e95bd36bbe3642169c31720ff0a1aed2` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:38:43.476355 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6eb0143-cc9e-42f4-8b85-1c16cdccd21a admin admin] RBAC: Authorizing `identity:delete_project(project_id=e95bd36bbe3642169c31720ff0a1aed2)` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:38:43.478378 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6eb0143-cc9e-42f4-8b85-1c16cdccd21a admin admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:38:43.499122 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1322/2655] 10.222.0.49 () {58 vars in 1223 bytes} [Wed Jun 15 10:38:43 2022] DELETE /identity/v3/users/c647d6ce46574dc987a4be6ff4a8d7f7 => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 10:38:43.502296 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d65242-c51b-429b-acff-4cfa15d63c6e None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:38:43.502844 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d65242-c51b-429b-acff-4cfa15d63c6e None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:38:43.503194 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d65242-c51b-429b-acff-4cfa15d63c6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:38:43.546988 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-26d65242-c51b-429b-acff-4cfa15d63c6e None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:38:43.548564 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-26d65242-c51b-429b-acff-4cfa15d63c6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:38:43.551123 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-a6eb0143-cc9e-42f4-8b85-1c16cdccd21a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e95bd36bbe3642169c31720ff0a1aed2', 'request_id': 'req-a6eb0143-cc9e-42f4-8b85-1c16cdccd21a'} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:38:43.563703 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-a6eb0143-cc9e-42f4-8b85-1c16cdccd21a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e95bd36bbe3642169c31720ff0a1aed2', 'request_id': 'req-a6eb0143-cc9e-42f4-8b85-1c16cdccd21a'} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:38:43.576604 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-a6eb0143-cc9e-42f4-8b85-1c16cdccd21a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e95bd36bbe3642169c31720ff0a1aed2', 'request_id': 'req-a6eb0143-cc9e-42f4-8b85-1c16cdccd21a'} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:38:43.594602 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-a6eb0143-cc9e-42f4-8b85-1c16cdccd21a admin admin] The token cache is being invalidate because project e95bd36bbe3642169c31720ff0a1aed2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80119) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 10:38:43.595250 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-a6eb0143-cc9e-42f4-8b85-1c16cdccd21a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:38:43.596270 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-a6eb0143-cc9e-42f4-8b85-1c16cdccd21a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:38:43.603047 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1334/2656] 10.222.0.49 () {58 vars in 1232 bytes} [Wed Jun 15 10:38:43 2022] DELETE /identity/v3/projects/e95bd36bbe3642169c31720ff0a1aed2 => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:38:43.606678 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1fc10e-d60f-4605-a67c-25cbd53cf600 None None] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:38:43.607369 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1fc10e-d60f-4605-a67c-25cbd53cf600 None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:38:43.608101 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1fc10e-d60f-4605-a67c-25cbd53cf600 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:38:43.644053 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.auth.core [None req-de1fc10e-d60f-4605-a67c-25cbd53cf600 None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:38:43.645403 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-de1fc10e-d60f-4605-a67c-25cbd53cf600 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:38:43.654872 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1335/2657] 10.222.0.49 () {58 vars in 957 bytes} [Wed Jun 15 10:38:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:38:43.655903 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1fc10e-d60f-4605-a67c-25cbd53cf600 None None] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:38:43.657287 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-de1fc10e-d60f-4605-a67c-25cbd53cf600 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:38:43.668097 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1323/2658] 10.222.0.49 () {58 vars in 957 bytes} [Wed Jun 15 10:38:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:38:43.671653 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1fc10e-d60f-4605-a67c-25cbd53cf600 None None] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:38:43.671777 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d65242-c51b-429b-acff-4cfa15d63c6e None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:38:43.676167 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-26d65242-c51b-429b-acff-4cfa15d63c6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:38:43.681833 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f326ba64-c9ce-4ff5-83be-20926d996db4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:38:43.682769 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f326ba64-c9ce-4ff5-83be-20926d996db4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:38:43.683194 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f326ba64-c9ce-4ff5-83be-20926d996db4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:38:43.683499 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f326ba64-c9ce-4ff5-83be-20926d996db4 admin admin] PATH_INFO: `/v3/users/a57b1195231846b7a72dd4c8a6482f53` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:38:43.686544 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f326ba64-c9ce-4ff5-83be-20926d996db4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a57b1195231846b7a72dd4c8a6482f53)` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:38:43.690579 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f326ba64-c9ce-4ff5-83be-20926d996db4 admin admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:38:43.768173 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-f326ba64-c9ce-4ff5-83be-20926d996db4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a57b1195231846b7a72dd4c8a6482f53'} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:38:43.778770 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-f326ba64-c9ce-4ff5-83be-20926d996db4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a57b1195231846b7a72dd4c8a6482f53'} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:38:43.780365 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-f326ba64-c9ce-4ff5-83be-20926d996db4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a57b1195231846b7a72dd4c8a6482f53'} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:38:43.781037 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-f326ba64-c9ce-4ff5-83be-20926d996db4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a57b1195231846b7a72dd4c8a6482f53'} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:38:43.800475 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-f326ba64-c9ce-4ff5-83be-20926d996db4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a57b1195231846b7a72dd4c8a6482f53'} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:38:43.810955 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d65242-c51b-429b-acff-4cfa15d63c6e None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:38:43.815867 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0819ef46-1742-4926-9112-fd8857a6dda7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0cb44a17d205431793e9a8791e477a32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '559b3e713f5844b6ae37151d56dd419f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:38:43.816552 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0819ef46-1742-4926-9112-fd8857a6dda7 service neutron] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:38:43.816850 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0819ef46-1742-4926-9112-fd8857a6dda7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:38:43.817131 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0819ef46-1742-4926-9112-fd8857a6dda7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:38:43.819288 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-0819ef46-1742-4926-9112-fd8857a6dda7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:38:43.825899 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1336/2659] 10.222.0.49 () {58 vars in 1223 bytes} [Wed Jun 15 10:38:43 2022] DELETE /identity/v3/users/a57b1195231846b7a72dd4c8a6482f53 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:38:43.830605 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fb2589-04ee-4706-9e48-d77e43b51d77 None None] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:38:43.830902 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fb2589-04ee-4706-9e48-d77e43b51d77 None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:38:43.831214 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fb2589-04ee-4706-9e48-d77e43b51d77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:38:43.833781 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0819ef46-1742-4926-9112-fd8857a6dda7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:38:43.835558 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0819ef46-1742-4926-9112-fd8857a6dda7 service neutron] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:38:43.848550 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1324/2660] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Jun 15 10:38:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 180 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:38:43.854452 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.auth.core [None req-79fb2589-04ee-4706-9e48-d77e43b51d77 None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:38:43.855696 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-79fb2589-04ee-4706-9e48-d77e43b51d77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:38:43.902365 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:38:43.953169 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1337/2661] 10.222.0.49 () {58 vars in 957 bytes} [Wed Jun 15 10:38:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:38:43.954831 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0819ef46-1742-4926-9112-fd8857a6dda7 service neutron] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:38:43.968526 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0819ef46-1742-4926-9112-fd8857a6dda7 service neutron] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:38:43.974459 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276d00de-fc05-41c9-9e1e-6602edec9817 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:38:43.977168 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276d00de-fc05-41c9-9e1e-6602edec9817 admin admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:38:43.977656 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276d00de-fc05-41c9-9e1e-6602edec9817 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:38:43.978103 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276d00de-fc05-41c9-9e1e-6602edec9817 admin admin] PATH_INFO: `/v3/domains` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:38:43.978930 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-276d00de-fc05-41c9-9e1e-6602edec9817 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:38:43.980286 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-276d00de-fc05-41c9-9e1e-6602edec9817 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:38:43.982119 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-276d00de-fc05-41c9-9e1e-6602edec9817 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:38:44.001981 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1325/2662] 10.222.0.49 () {58 vars in 1152 bytes} [Wed Jun 15 10:38:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:38:44.004387 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79fb2589-04ee-4706-9e48-d77e43b51d77 None None] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:38:44.021262 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79fb2589-04ee-4706-9e48-d77e43b51d77 None None] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:38:44.028620 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35613c2-08f2-41ff-ae28-919c6da3f842 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:38:44.029453 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35613c2-08f2-41ff-ae28-919c6da3f842 admin admin] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:38:44.029747 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35613c2-08f2-41ff-ae28-919c6da3f842 admin admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:38:44.030196 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35613c2-08f2-41ff-ae28-919c6da3f842 admin admin] PATH_INFO: `/v3/projects` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:38:44.030920 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b35613c2-08f2-41ff-ae28-919c6da3f842 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:38:44.043666 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b35613c2-08f2-41ff-ae28-919c6da3f842 admin admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:38:44.110342 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1338/2663] 10.222.0.49 () {60 vars in 1152 bytes} [Wed Jun 15 10:38:44 2022] POST /identity/v3/projects => generated 381 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 10:38:44.113973 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276d00de-fc05-41c9-9e1e-6602edec9817 admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:38:44.131822 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276d00de-fc05-41c9-9e1e-6602edec9817 admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:38:44.236796 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28291d70-5096-4002-9849-be3eb3085eb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:38:44.237497 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28291d70-5096-4002-9849-be3eb3085eb6 admin admin] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:38:44.237784 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28291d70-5096-4002-9849-be3eb3085eb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:38:44.238117 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28291d70-5096-4002-9849-be3eb3085eb6 admin admin] PATH_INFO: `/v3/users` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:38:44.239911 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28291d70-5096-4002-9849-be3eb3085eb6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:38:44.241479 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28291d70-5096-4002-9849-be3eb3085eb6 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:38:44.289504 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1326/2664] 10.222.0.49 () {60 vars in 1143 bytes} [Wed Jun 15 10:38:44 2022] POST /identity/v3/users => generated 291 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 10:38:44.290411 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35613c2-08f2-41ff-ae28-919c6da3f842 admin admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:38:44.308508 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35613c2-08f2-41ff-ae28-919c6da3f842 admin admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:38:44.315337 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e716989-563a-450a-b7ab-023c9d467e35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:38:44.316215 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e716989-563a-450a-b7ab-023c9d467e35 admin admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:38:44.316511 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e716989-563a-450a-b7ab-023c9d467e35 admin admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:38:44.316793 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e716989-563a-450a-b7ab-023c9d467e35 admin admin] PATH_INFO: `/v3/roles` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:38:44.317545 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e716989-563a-450a-b7ab-023c9d467e35 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:38:44.319235 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e716989-563a-450a-b7ab-023c9d467e35 admin admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:38:44.351594 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1339/2665] 10.222.0.49 () {58 vars in 1121 bytes} [Wed Jun 15 10:38:44 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 10:38:44.352775 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28291d70-5096-4002-9849-be3eb3085eb6 admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:38:44.367037 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:DELETE=2,keystone:SELECT=95,keystone:INSERT=3 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:38:44.374146 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28291d70-5096-4002-9849-be3eb3085eb6 admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:38:44.388108 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a2dbcd-23a3-4109-a48f-4c5946492c50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:38:44.389604 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a2dbcd-23a3-4109-a48f-4c5946492c50 admin admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:38:44.390074 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a2dbcd-23a3-4109-a48f-4c5946492c50 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:38:44.390530 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a2dbcd-23a3-4109-a48f-4c5946492c50 admin admin] PATH_INFO: `/v3/roles` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:38:44.391547 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a2dbcd-23a3-4109-a48f-4c5946492c50 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:38:44.393346 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a2dbcd-23a3-4109-a48f-4c5946492c50 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:38:44.417231 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1327/2666] 10.222.0.49 () {58 vars in 1121 bytes} [Wed Jun 15 10:38:44 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 10:38:44.419397 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e716989-563a-450a-b7ab-023c9d467e35 admin admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:38:44.432097 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a2dbcd-23a3-4109-a48f-4c5946492c50 admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:38:44.435871 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-53a2dbcd-23a3-4109-a48f-4c5946492c50 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:38:44.439035 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e716989-563a-450a-b7ab-023c9d467e35 admin admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:38:44.453276 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3ad85b-6a25-4c6b-ab97-f876385bafff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:38:44.454733 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3ad85b-6a25-4c6b-ab97-f876385bafff admin admin] REQUEST_METHOD: `PUT` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:38:44.455017 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3ad85b-6a25-4c6b-ab97-f876385bafff admin admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:38:44.455336 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3ad85b-6a25-4c6b-ab97-f876385bafff admin admin] PATH_INFO: `/v3/projects/4a0f117ef57a4c08bdf37604b76a6793/users/666c90cfe3ce4e53ad4ab1dcfd2be37d/roles/6876e57cc51c448097ab109d35b2a3da` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:38:44.458257 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a2dbcd-23a3-4109-a48f-4c5946492c50 admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:38:44.471357 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59ca910-d176-4fda-b953-731e5adb92c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:38:44.472878 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59ca910-d176-4fda-b953-731e5adb92c0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:38:44.474019 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59ca910-d176-4fda-b953-731e5adb92c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:38:44.476839 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59ca910-d176-4fda-b953-731e5adb92c0 admin admin] PATH_INFO: `/v3/projects/347fc73d3eb84c4e89bf0cf160bc52af` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:38:44.477968 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c3ad85b-6a25-4c6b-ab97-f876385bafff admin admin] RBAC: Authorizing `identity:create_grant(project_id=4a0f117ef57a4c08bdf37604b76a6793, user_id=666c90cfe3ce4e53ad4ab1dcfd2be37d, role_id=6876e57cc51c448097ab109d35b2a3da)` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:38:44.480757 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c3ad85b-6a25-4c6b-ab97-f876385bafff admin admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:38:44.483644 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a59ca910-d176-4fda-b953-731e5adb92c0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=347fc73d3eb84c4e89bf0cf160bc52af)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:38:44.487964 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a59ca910-d176-4fda-b953-731e5adb92c0 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:38:44.508103 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1340/2667] 10.222.0.49 () {60 vars in 1482 bytes} [Wed Jun 15 10:38:44 2022] PUT /identity/v3/projects/4a0f117ef57a4c08bdf37604b76a6793/users/666c90cfe3ce4e53ad4ab1dcfd2be37d/roles/6876e57cc51c448097ab109d35b2a3da => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:38:44.511859 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa503d8-ecaf-4a59-9411-ac9da8b27242 None None] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:38:44.512256 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa503d8-ecaf-4a59-9411-ac9da8b27242 None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:38:44.512587 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa503d8-ecaf-4a59-9411-ac9da8b27242 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:38:44.550431 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.auth.core [None req-7fa503d8-ecaf-4a59-9411-ac9da8b27242 None None] MFA Rules not processed for user `666c90cfe3ce4e53ad4ab1dcfd2be37d`. Rule list: `[]` (Enabled: `True`). {{(pid=80119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:38:44.551919 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-7fa503d8-ecaf-4a59-9411-ac9da8b27242 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:38:44.615736 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-a59ca910-d176-4fda-b953-731e5adb92c0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '347fc73d3eb84c4e89bf0cf160bc52af', 'request_id': 'req-a59ca910-d176-4fda-b953-731e5adb92c0'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:38:44.650603 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-a59ca910-d176-4fda-b953-731e5adb92c0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '347fc73d3eb84c4e89bf0cf160bc52af', 'request_id': 'req-a59ca910-d176-4fda-b953-731e5adb92c0'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:38:44.658666 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-a59ca910-d176-4fda-b953-731e5adb92c0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '347fc73d3eb84c4e89bf0cf160bc52af', 'request_id': 'req-a59ca910-d176-4fda-b953-731e5adb92c0'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:38:44.675989 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-a59ca910-d176-4fda-b953-731e5adb92c0 admin admin] The token cache is being invalidate because project 347fc73d3eb84c4e89bf0cf160bc52af was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80120) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 10:38:44.676617 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-a59ca910-d176-4fda-b953-731e5adb92c0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:38:44.676720 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1341/2668] 10.222.0.49 () {58 vars in 957 bytes} [Wed Jun 15 10:38:44 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:38:44.677627 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa503d8-ecaf-4a59-9411-ac9da8b27242 None None] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:38:44.678976 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-7fa503d8-ecaf-4a59-9411-ac9da8b27242 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:38:44.679811 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-a59ca910-d176-4fda-b953-731e5adb92c0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:38:44.681633 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1328/2669] 10.222.0.49 () {58 vars in 1232 bytes} [Wed Jun 15 10:38:44 2022] DELETE /identity/v3/projects/347fc73d3eb84c4e89bf0cf160bc52af => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:38:44.682987 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db946bf5-b8ad-44dc-9544-b3cd06184193 None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:38:44.683333 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db946bf5-b8ad-44dc-9544-b3cd06184193 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:38:44.683677 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db946bf5-b8ad-44dc-9544-b3cd06184193 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:38:44.714463 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-db946bf5-b8ad-44dc-9544-b3cd06184193 None None] MFA Rules not processed for user `61bc9ad2c9444fc297eb4b5902a67b98`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:38:44.715874 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-db946bf5-b8ad-44dc-9544-b3cd06184193 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:38:44.770341 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa503d8-ecaf-4a59-9411-ac9da8b27242 None None] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:38:44.792746 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1329/2670] 10.222.0.49 () {58 vars in 957 bytes} [Wed Jun 15 10:38:44 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:38:44.793593 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db946bf5-b8ad-44dc-9544-b3cd06184193 None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:38:44.795244 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-db946bf5-b8ad-44dc-9544-b3cd06184193 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:38:44.795717 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:38:44.851488 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38e9b34-a4d2-42e2-8c16-b24296f16a72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:38:44.852421 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38e9b34-a4d2-42e2-8c16-b24296f16a72 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:38:44.852713 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38e9b34-a4d2-42e2-8c16-b24296f16a72 admin admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:38:44.852991 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38e9b34-a4d2-42e2-8c16-b24296f16a72 admin admin] PATH_INFO: `/v3/projects/25994ed2792548f099025cfe68c94d97` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:38:44.854384 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c38e9b34-a4d2-42e2-8c16-b24296f16a72 admin admin] RBAC: Authorizing `identity:delete_project(project_id=25994ed2792548f099025cfe68c94d97)` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:38:44.856361 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c38e9b34-a4d2-42e2-8c16-b24296f16a72 admin admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:38:44.873461 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db946bf5-b8ad-44dc-9544-b3cd06184193 None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:38:44.878417 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e564ccfb-e547-4fa8-8743-049f7f12ec7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:38:44.879123 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e564ccfb-e547-4fa8-8743-049f7f12ec7d admin admin] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:38:44.879746 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e564ccfb-e547-4fa8-8743-049f7f12ec7d admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:38:44.879746 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e564ccfb-e547-4fa8-8743-049f7f12ec7d admin admin] PATH_INFO: `/v3/projects` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:38:44.880440 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e564ccfb-e547-4fa8-8743-049f7f12ec7d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:38:44.882424 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e564ccfb-e547-4fa8-8743-049f7f12ec7d admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:38:44.894043 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:38:44.908941 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-c38e9b34-a4d2-42e2-8c16-b24296f16a72 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '25994ed2792548f099025cfe68c94d97', 'request_id': 'req-c38e9b34-a4d2-42e2-8c16-b24296f16a72'} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:38:44.915173 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1330/2671] 10.222.0.49 () {60 vars in 1152 bytes} [Wed Jun 15 10:38:44 2022] POST /identity/v3/projects => generated 381 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 10:38:44.916725 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4d082d-2ed3-47a2-bfe2-defeddb9277d None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:38:44.917048 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4d082d-2ed3-47a2-bfe2-defeddb9277d None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:38:44.917349 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4d082d-2ed3-47a2-bfe2-defeddb9277d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:38:44.917567 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-c38e9b34-a4d2-42e2-8c16-b24296f16a72 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '25994ed2792548f099025cfe68c94d97', 'request_id': 'req-c38e9b34-a4d2-42e2-8c16-b24296f16a72'} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:38:44.925754 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-c38e9b34-a4d2-42e2-8c16-b24296f16a72 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '25994ed2792548f099025cfe68c94d97', 'request_id': 'req-c38e9b34-a4d2-42e2-8c16-b24296f16a72'} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:38:44.939453 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-0e4d082d-2ed3-47a2-bfe2-defeddb9277d None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:38:44.939730 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-c38e9b34-a4d2-42e2-8c16-b24296f16a72 admin admin] The token cache is being invalidate because project 25994ed2792548f099025cfe68c94d97 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80119) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 10:38:44.940008 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-c38e9b34-a4d2-42e2-8c16-b24296f16a72 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:38:44.940695 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-c38e9b34-a4d2-42e2-8c16-b24296f16a72 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:38:44.941160 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-0e4d082d-2ed3-47a2-bfe2-defeddb9277d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:38:44.942492 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1342/2672] 10.222.0.49 () {58 vars in 1232 bytes} [Wed Jun 15 10:38:44 2022] DELETE /identity/v3/projects/25994ed2792548f099025cfe68c94d97 => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:38:44.943031 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38e9b34-a4d2-42e2-8c16-b24296f16a72 admin admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:38:44.944703 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-c38e9b34-a4d2-42e2-8c16-b24296f16a72 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:38:45.009817 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1331/2673] 10.222.0.49 () {58 vars in 957 bytes} [Wed Jun 15 10:38:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:38:45.011973 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4d082d-2ed3-47a2-bfe2-defeddb9277d None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:38:45.017743 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-0e4d082d-2ed3-47a2-bfe2-defeddb9277d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:38:45.018866 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38e9b34-a4d2-42e2-8c16-b24296f16a72 admin admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:38:45.023845 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54e3956-9710-4aa5-babd-e78a1f899a35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:38:45.024785 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54e3956-9710-4aa5-babd-e78a1f899a35 admin admin] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:38:45.025073 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54e3956-9710-4aa5-babd-e78a1f899a35 admin admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:38:45.025496 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54e3956-9710-4aa5-babd-e78a1f899a35 admin admin] PATH_INFO: `/v3/projects` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:38:45.026205 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e54e3956-9710-4aa5-babd-e78a1f899a35 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:38:45.028253 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e54e3956-9710-4aa5-babd-e78a1f899a35 admin admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:38:45.032469 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4d082d-2ed3-47a2-bfe2-defeddb9277d None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:38:45.038246 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c65add5-df83-4041-b55a-186ad137c7ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:38:45.039427 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c65add5-df83-4041-b55a-186ad137c7ce admin admin] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:38:45.039711 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c65add5-df83-4041-b55a-186ad137c7ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:38:45.039997 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c65add5-df83-4041-b55a-186ad137c7ce admin admin] PATH_INFO: `/v3/users` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:38:45.040702 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c65add5-df83-4041-b55a-186ad137c7ce admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:38:45.042643 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c65add5-df83-4041-b55a-186ad137c7ce admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:38:45.061407 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1343/2674] 10.222.0.49 () {60 vars in 1152 bytes} [Wed Jun 15 10:38:44 2022] POST /identity/v3/projects => generated 379 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 10:38:45.061407 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7d744a-a4bd-47c8-b94f-11ec7c311282 None None] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:38:45.061407 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7d744a-a4bd-47c8-b94f-11ec7c311282 None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:38:45.061407 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7d744a-a4bd-47c8-b94f-11ec7c311282 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:38:45.066558 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1332/2675] 10.222.0.49 () {60 vars in 1143 bytes} [Wed Jun 15 10:38:45 2022] POST /identity/v3/users => generated 297 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 10:38:45.068000 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a04cf8d-c00a-4f40-b3dd-f61ece4cea99 None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:38:45.068597 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a04cf8d-c00a-4f40-b3dd-f61ece4cea99 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:38:45.068899 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a04cf8d-c00a-4f40-b3dd-f61ece4cea99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:38:45.082626 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.auth.core [None req-bb7d744a-a4bd-47c8-b94f-11ec7c311282 None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:38:45.084276 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-bb7d744a-a4bd-47c8-b94f-11ec7c311282 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:38:45.089633 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-8a04cf8d-c00a-4f40-b3dd-f61ece4cea99 None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:38:45.091118 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-8a04cf8d-c00a-4f40-b3dd-f61ece4cea99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:38:45.118220 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:38:45.154261 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1344/2676] 10.222.0.49 () {58 vars in 957 bytes} [Wed Jun 15 10:38:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:38:45.155218 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7d744a-a4bd-47c8-b94f-11ec7c311282 None None] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:38:45.170296 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1333/2677] 10.222.0.49 () {58 vars in 957 bytes} [Wed Jun 15 10:38:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:38:45.171986 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7d744a-a4bd-47c8-b94f-11ec7c311282 None None] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:38:45.172453 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a04cf8d-c00a-4f40-b3dd-f61ece4cea99 None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:38:45.177172 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3916f9b4-acce-4dfa-8552-6dc014a4c66b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:38:45.177871 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3916f9b4-acce-4dfa-8552-6dc014a4c66b admin admin] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:38:45.178163 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3916f9b4-acce-4dfa-8552-6dc014a4c66b admin admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:38:45.178447 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3916f9b4-acce-4dfa-8552-6dc014a4c66b admin admin] PATH_INFO: `/v3/users` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:38:45.180860 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3916f9b4-acce-4dfa-8552-6dc014a4c66b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:38:45.183758 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3916f9b4-acce-4dfa-8552-6dc014a4c66b admin admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:38:45.185074 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a04cf8d-c00a-4f40-b3dd-f61ece4cea99 None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:38:45.191944 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274a66b1-b724-47dc-8b4e-d0eb216622ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:38:45.192807 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274a66b1-b724-47dc-8b4e-d0eb216622ae admin admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:38:45.193097 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274a66b1-b724-47dc-8b4e-d0eb216622ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:38:45.193376 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274a66b1-b724-47dc-8b4e-d0eb216622ae admin admin] PATH_INFO: `/v3/roles` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:38:45.194116 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-274a66b1-b724-47dc-8b4e-d0eb216622ae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:38:45.196042 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-274a66b1-b724-47dc-8b4e-d0eb216622ae admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:38:45.209564 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1334/2678] 10.222.0.49 () {58 vars in 1121 bytes} [Wed Jun 15 10:38:45 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 10:38:45.211644 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3df696-fe8e-4439-9c2d-32a18fd2879e None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:38:45.211969 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3df696-fe8e-4439-9c2d-32a18fd2879e None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:38:45.212216 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3df696-fe8e-4439-9c2d-32a18fd2879e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:38:45.227031 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1345/2679] 10.222.0.49 () {60 vars in 1143 bytes} [Wed Jun 15 10:38:45 2022] POST /identity/v3/users => generated 296 bytes in 69 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 10:38:45.227031 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3916f9b4-acce-4dfa-8552-6dc014a4c66b admin admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:38:45.236921 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3916f9b4-acce-4dfa-8552-6dc014a4c66b admin admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:38:45.242395 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e3abf9-0924-40d2-a2da-2d3453959f98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:38:45.243080 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e3abf9-0924-40d2-a2da-2d3453959f98 admin admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:38:45.243406 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e3abf9-0924-40d2-a2da-2d3453959f98 admin admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:38:45.243710 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e3abf9-0924-40d2-a2da-2d3453959f98 admin admin] PATH_INFO: `/v3/domains` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:38:45.244447 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e3abf9-0924-40d2-a2da-2d3453959f98 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:38:45.244653 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-5b3df696-fe8e-4439-9c2d-32a18fd2879e None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:38:45.244778 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e3abf9-0924-40d2-a2da-2d3453959f98 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:38:45.245883 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-5b3df696-fe8e-4439-9c2d-32a18fd2879e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:38:45.246363 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e3abf9-0924-40d2-a2da-2d3453959f98 admin admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:38:45.255948 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1335/2680] 10.222.0.49 () {58 vars in 957 bytes} [Wed Jun 15 10:38:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:38:45.256757 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3df696-fe8e-4439-9c2d-32a18fd2879e None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:38:45.264084 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1346/2681] 10.222.0.49 () {58 vars in 1152 bytes} [Wed Jun 15 10:38:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:38:45.264084 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e3abf9-0924-40d2-a2da-2d3453959f98 admin admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:38:45.267394 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3df696-fe8e-4439-9c2d-32a18fd2879e None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:38:45.273425 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c113ffca-f314-45b1-bc71-670ac5e049d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:38:45.273804 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e3abf9-0924-40d2-a2da-2d3453959f98 admin admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:38:45.274606 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c113ffca-f314-45b1-bc71-670ac5e049d0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:38:45.274893 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c113ffca-f314-45b1-bc71-670ac5e049d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:38:45.275213 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c113ffca-f314-45b1-bc71-670ac5e049d0 admin admin] PATH_INFO: `/v3/projects/0fd3f208877a490baf55581ebbc5a701/users/36dc549976e940f1b42b30580eafde65/roles/8051174e7a7947d5b991ebd793c2e543` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:38:45.279254 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9675456-fea3-4861-ad93-2915d892c80b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:38:45.279945 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9675456-fea3-4861-ad93-2915d892c80b admin admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:38:45.280228 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9675456-fea3-4861-ad93-2915d892c80b admin admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:38:45.280504 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9675456-fea3-4861-ad93-2915d892c80b admin admin] PATH_INFO: `/v3/roles` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:38:45.281238 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9675456-fea3-4861-ad93-2915d892c80b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:38:45.282924 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9675456-fea3-4861-ad93-2915d892c80b admin admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:38:45.291597 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c113ffca-f314-45b1-bc71-670ac5e049d0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0fd3f208877a490baf55581ebbc5a701, user_id=36dc549976e940f1b42b30580eafde65, role_id=8051174e7a7947d5b991ebd793c2e543)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:38:45.293537 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1347/2682] 10.222.0.49 () {58 vars in 1121 bytes} [Wed Jun 15 10:38:45 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 10:38:45.294345 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9675456-fea3-4861-ad93-2915d892c80b admin admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:38:45.294595 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c113ffca-f314-45b1-bc71-670ac5e049d0 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:38:45.305749 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9675456-fea3-4861-ad93-2915d892c80b admin admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:38:45.315565 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1336/2683] 10.222.0.49 () {60 vars in 1482 bytes} [Wed Jun 15 10:38:45 2022] PUT /identity/v3/projects/0fd3f208877a490baf55581ebbc5a701/users/36dc549976e940f1b42b30580eafde65/roles/8051174e7a7947d5b991ebd793c2e543 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:38:45.317048 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c113ffca-f314-45b1-bc71-670ac5e049d0 admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:38:45.366102 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:38:45.375354 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc136f1a-7709-4fc4-a624-66a30f46a0e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:38:45.376635 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc136f1a-7709-4fc4-a624-66a30f46a0e2 admin admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:38:45.376819 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc136f1a-7709-4fc4-a624-66a30f46a0e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:38:45.377087 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc136f1a-7709-4fc4-a624-66a30f46a0e2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:38:45.377847 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc136f1a-7709-4fc4-a624-66a30f46a0e2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:38:45.378159 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc136f1a-7709-4fc4-a624-66a30f46a0e2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:38:45.379796 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc136f1a-7709-4fc4-a624-66a30f46a0e2 admin admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:38:45.391508 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1348/2684] 10.222.0.49 () {58 vars in 1152 bytes} [Wed Jun 15 10:38:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:38:45.392294 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc136f1a-7709-4fc4-a624-66a30f46a0e2 admin admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:38:45.395583 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c113ffca-f314-45b1-bc71-670ac5e049d0 admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:38:45.400683 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67a77cd-7fac-4dd2-aed9-49704b93b97f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:38:45.401377 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67a77cd-7fac-4dd2-aed9-49704b93b97f admin admin] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:38:45.401663 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67a77cd-7fac-4dd2-aed9-49704b93b97f admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:38:45.401945 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67a77cd-7fac-4dd2-aed9-49704b93b97f admin admin] PATH_INFO: `/v3/projects` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:38:45.402664 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a67a77cd-7fac-4dd2-aed9-49704b93b97f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:38:45.404649 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a67a77cd-7fac-4dd2-aed9-49704b93b97f admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:38:45.404770 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc136f1a-7709-4fc4-a624-66a30f46a0e2 admin admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:38:45.409337 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8693bfc3-7b62-4fd3-a72a-ea25a19f3fa1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:38:45.410194 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8693bfc3-7b62-4fd3-a72a-ea25a19f3fa1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:38:45.410474 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8693bfc3-7b62-4fd3-a72a-ea25a19f3fa1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:38:45.410745 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8693bfc3-7b62-4fd3-a72a-ea25a19f3fa1 admin admin] PATH_INFO: `/v3/projects/ad67bfe861ba491c89827c72982ea8c2/users/43929157365d4fad979f66e227572393/roles/8051174e7a7947d5b991ebd793c2e543` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:38:45.426869 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8693bfc3-7b62-4fd3-a72a-ea25a19f3fa1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ad67bfe861ba491c89827c72982ea8c2, user_id=43929157365d4fad979f66e227572393, role_id=8051174e7a7947d5b991ebd793c2e543)` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:38:45.429511 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8693bfc3-7b62-4fd3-a72a-ea25a19f3fa1 admin admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:38:45.435210 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1337/2685] 10.222.0.49 () {60 vars in 1152 bytes} [Wed Jun 15 10:38:45 2022] POST /identity/v3/projects => generated 373 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 10:38:45.435210 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67a77cd-7fac-4dd2-aed9-49704b93b97f admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:38:45.444974 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1349/2686] 10.222.0.49 () {60 vars in 1482 bytes} [Wed Jun 15 10:38:45 2022] PUT /identity/v3/projects/ad67bfe861ba491c89827c72982ea8c2/users/43929157365d4fad979f66e227572393/roles/8051174e7a7947d5b991ebd793c2e543 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:38:45.447316 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8693bfc3-7b62-4fd3-a72a-ea25a19f3fa1 admin admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:38:45.503764 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67a77cd-7fac-4dd2-aed9-49704b93b97f admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:38:45.506511 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8693bfc3-7b62-4fd3-a72a-ea25a19f3fa1 admin admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:38:45.508485 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c055ae4-af79-4d30-baae-db1db62f4847 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:38:45.509183 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c055ae4-af79-4d30-baae-db1db62f4847 admin admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:38:45.509605 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c055ae4-af79-4d30-baae-db1db62f4847 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:38:45.510075 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c055ae4-af79-4d30-baae-db1db62f4847 admin admin] PATH_INFO: `/v3/roles` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:38:45.510764 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c055ae4-af79-4d30-baae-db1db62f4847 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:38:45.512317 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90502b5-dcd7-4ecb-9f2e-b25cd95a6bc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:38:45.512455 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c055ae4-af79-4d30-baae-db1db62f4847 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:38:45.512987 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90502b5-dcd7-4ecb-9f2e-b25cd95a6bc4 admin admin] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:38:45.513269 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90502b5-dcd7-4ecb-9f2e-b25cd95a6bc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:38:45.513545 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90502b5-dcd7-4ecb-9f2e-b25cd95a6bc4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:38:45.514235 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90502b5-dcd7-4ecb-9f2e-b25cd95a6bc4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:38:45.516197 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90502b5-dcd7-4ecb-9f2e-b25cd95a6bc4 admin admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:38:45.520689 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:38:45.527964 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1338/2687] 10.222.0.49 () {58 vars in 1121 bytes} [Wed Jun 15 10:38:45 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 10:38:45.527964 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c055ae4-af79-4d30-baae-db1db62f4847 admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:38:45.539288 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c055ae4-af79-4d30-baae-db1db62f4847 admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:38:45.543779 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1350/2688] 10.222.0.49 () {60 vars in 1152 bytes} [Wed Jun 15 10:38:45 2022] POST /identity/v3/projects => generated 375 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 10:38:45.543913 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd6f6c3-bdbc-4ef3-bc29-172f7a61d54a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:38:45.544580 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd6f6c3-bdbc-4ef3-bc29-172f7a61d54a admin admin] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:38:45.544777 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90502b5-dcd7-4ecb-9f2e-b25cd95a6bc4 admin admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:38:45.544886 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd6f6c3-bdbc-4ef3-bc29-172f7a61d54a admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:38:45.545142 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd6f6c3-bdbc-4ef3-bc29-172f7a61d54a admin admin] PATH_INFO: `/v3/users` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:38:45.545803 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd6f6c3-bdbc-4ef3-bc29-172f7a61d54a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:38:45.548295 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd6f6c3-bdbc-4ef3-bc29-172f7a61d54a admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:38:45.570236 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1339/2689] 10.222.0.49 () {60 vars in 1143 bytes} [Wed Jun 15 10:38:45 2022] POST /identity/v3/users => generated 287 bytes in 43 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 10:38:45.571021 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd6f6c3-bdbc-4ef3-bc29-172f7a61d54a admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:38:45.580489 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd6f6c3-bdbc-4ef3-bc29-172f7a61d54a admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:38:45.609548 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90502b5-dcd7-4ecb-9f2e-b25cd95a6bc4 admin admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:38:45.615384 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd98fb8-e877-4e1c-a57a-aed0df82c442 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:38:45.616050 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd98fb8-e877-4e1c-a57a-aed0df82c442 admin admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:38:45.616380 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd98fb8-e877-4e1c-a57a-aed0df82c442 admin admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:38:45.616661 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd98fb8-e877-4e1c-a57a-aed0df82c442 admin admin] PATH_INFO: `/v3/roles` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:38:45.617388 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd98fb8-e877-4e1c-a57a-aed0df82c442 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:38:45.619066 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd98fb8-e877-4e1c-a57a-aed0df82c442 admin admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:38:45.629402 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1351/2690] 10.222.0.49 () {58 vars in 1121 bytes} [Wed Jun 15 10:38:45 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 10:38:45.630372 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd98fb8-e877-4e1c-a57a-aed0df82c442 admin admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:38:45.641005 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e39eac-8126-4db3-abd3-65119c267768 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:38:45.641900 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e39eac-8126-4db3-abd3-65119c267768 admin admin] REQUEST_METHOD: `PUT` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:38:45.642179 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e39eac-8126-4db3-abd3-65119c267768 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:38:45.642455 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e39eac-8126-4db3-abd3-65119c267768 admin admin] PATH_INFO: `/v3/projects/0fd3f208877a490baf55581ebbc5a701/users/36dc549976e940f1b42b30580eafde65/roles/8051174e7a7947d5b991ebd793c2e543` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:38:45.642605 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd98fb8-e877-4e1c-a57a-aed0df82c442 admin admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:38:45.644456 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e39eac-8126-4db3-abd3-65119c267768 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0fd3f208877a490baf55581ebbc5a701, user_id=36dc549976e940f1b42b30580eafde65, role_id=8051174e7a7947d5b991ebd793c2e543)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:38:45.647112 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e39eac-8126-4db3-abd3-65119c267768 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:38:45.647223 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51e7a1d-322c-4a20-a225-757c0fc0950b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:38:45.647883 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51e7a1d-322c-4a20-a225-757c0fc0950b admin admin] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:38:45.648173 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51e7a1d-322c-4a20-a225-757c0fc0950b admin admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:38:45.648792 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51e7a1d-322c-4a20-a225-757c0fc0950b admin admin] PATH_INFO: `/v3/users` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:38:45.649745 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f51e7a1d-322c-4a20-a225-757c0fc0950b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:38:45.651794 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f51e7a1d-322c-4a20-a225-757c0fc0950b admin admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:38:45.658298 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1340/2691] 10.222.0.49 () {60 vars in 1482 bytes} [Wed Jun 15 10:38:45 2022] PUT /identity/v3/projects/0fd3f208877a490baf55581ebbc5a701/users/36dc549976e940f1b42b30580eafde65/roles/8051174e7a7947d5b991ebd793c2e543 => generated 0 bytes in 88 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:38:45.659211 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e39eac-8126-4db3-abd3-65119c267768 admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:38:45.677374 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1352/2692] 10.222.0.49 () {60 vars in 1143 bytes} [Wed Jun 15 10:38:45 2022] POST /identity/v3/users => generated 288 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 10:38:45.677374 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51e7a1d-322c-4a20-a225-757c0fc0950b admin admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:38:45.688025 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51e7a1d-322c-4a20-a225-757c0fc0950b admin admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:38:45.717132 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:38:45.754192 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e39eac-8126-4db3-abd3-65119c267768 admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:38:45.758765 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8760585-6b9e-467b-8d8f-122e2f7f2815 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:38:45.759725 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8760585-6b9e-467b-8d8f-122e2f7f2815 admin admin] REQUEST_METHOD: `PUT` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:38:45.759887 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7e15c8-1863-4688-994a-c3a58d5c4f90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:38:45.760014 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8760585-6b9e-467b-8d8f-122e2f7f2815 admin admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:38:45.760274 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8760585-6b9e-467b-8d8f-122e2f7f2815 admin admin] PATH_INFO: `/v3/projects/ad67bfe861ba491c89827c72982ea8c2/users/43929157365d4fad979f66e227572393/roles/8051174e7a7947d5b991ebd793c2e543` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:38:45.760970 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7e15c8-1863-4688-994a-c3a58d5c4f90 admin admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:38:45.761433 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7e15c8-1863-4688-994a-c3a58d5c4f90 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:38:45.761818 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7e15c8-1863-4688-994a-c3a58d5c4f90 admin admin] PATH_INFO: `/v3/roles` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:38:45.762498 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8760585-6b9e-467b-8d8f-122e2f7f2815 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ad67bfe861ba491c89827c72982ea8c2, user_id=43929157365d4fad979f66e227572393, role_id=8051174e7a7947d5b991ebd793c2e543)` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:38:45.762753 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a7e15c8-1863-4688-994a-c3a58d5c4f90 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:38:45.764615 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a7e15c8-1863-4688-994a-c3a58d5c4f90 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:38:45.765572 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8760585-6b9e-467b-8d8f-122e2f7f2815 admin admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:38:45.785558 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1341/2693] 10.222.0.49 () {58 vars in 1121 bytes} [Wed Jun 15 10:38:45 2022] GET /identity/v3/roles => generated 1420 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 10:38:45.785558 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b37552-0723-4818-ab9e-a2eb34d4f993 None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:38:45.785558 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b37552-0723-4818-ab9e-a2eb34d4f993 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:38:45.785558 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b37552-0723-4818-ab9e-a2eb34d4f993 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:38:45.792807 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1353/2694] 10.222.0.49 () {60 vars in 1482 bytes} [Wed Jun 15 10:38:45 2022] PUT /identity/v3/projects/ad67bfe861ba491c89827c72982ea8c2/users/43929157365d4fad979f66e227572393/roles/8051174e7a7947d5b991ebd793c2e543 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:38:45.792807 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8760585-6b9e-467b-8d8f-122e2f7f2815 admin admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:38:45.831649 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-a2b37552-0723-4818-ab9e-a2eb34d4f993 None None] MFA Rules not processed for user `36dc549976e940f1b42b30580eafde65`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:38:45.833774 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-a2b37552-0723-4818-ab9e-a2eb34d4f993 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:38:45.882836 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8760585-6b9e-467b-8d8f-122e2f7f2815 admin admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:38:45.888827 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a47968-d9bb-4524-a162-3a0e74432b1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:38:45.889526 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a47968-d9bb-4524-a162-3a0e74432b1b admin admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:38:45.890116 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a47968-d9bb-4524-a162-3a0e74432b1b admin admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:38:45.890561 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a47968-d9bb-4524-a162-3a0e74432b1b admin admin] PATH_INFO: `/v3/roles` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:38:45.891577 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a47968-d9bb-4524-a162-3a0e74432b1b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:38:45.893443 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a47968-d9bb-4524-a162-3a0e74432b1b admin admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:38:45.895706 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:38:45.919725 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1354/2695] 10.222.0.49 () {58 vars in 1121 bytes} [Wed Jun 15 10:38:45 2022] GET /identity/v3/roles => generated 1420 bytes in 130 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 10:38:45.924734 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a47968-d9bb-4524-a162-3a0e74432b1b admin admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:38:45.936053 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a47968-d9bb-4524-a162-3a0e74432b1b admin admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:38:45.946582 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca00244-4c56-4ed1-8405-1e4da1d16477 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:38:45.947606 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca00244-4c56-4ed1-8405-1e4da1d16477 admin admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:38:45.947980 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca00244-4c56-4ed1-8405-1e4da1d16477 admin admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:38:45.948346 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca00244-4c56-4ed1-8405-1e4da1d16477 admin admin] PATH_INFO: `/v3/roles` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:38:45.949305 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca00244-4c56-4ed1-8405-1e4da1d16477 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:38:45.951026 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca00244-4c56-4ed1-8405-1e4da1d16477 admin admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:38:45.965903 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1342/2696] 10.222.0.49 () {58 vars in 957 bytes} [Wed Jun 15 10:38:45 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:38:45.965903 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b37552-0723-4818-ab9e-a2eb34d4f993 None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:38:45.982102 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1355/2697] 10.222.0.49 () {58 vars in 1121 bytes} [Wed Jun 15 10:38:45 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 10:38:45.983408 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca00244-4c56-4ed1-8405-1e4da1d16477 admin admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:38:45.985393 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-3ca00244-4c56-4ed1-8405-1e4da1d16477 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:38:45.986592 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b37552-0723-4818-ab9e-a2eb34d4f993 None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:38:45.992923 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7242bf-abe9-424a-b30d-5b326482f4dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:38:45.994210 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7242bf-abe9-424a-b30d-5b326482f4dc admin admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:38:45.994817 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7242bf-abe9-424a-b30d-5b326482f4dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:38:45.995330 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7242bf-abe9-424a-b30d-5b326482f4dc admin admin] PATH_INFO: `/v3/roles` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:38:45.996911 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7242bf-abe9-424a-b30d-5b326482f4dc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:38:46.001288 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7242bf-abe9-424a-b30d-5b326482f4dc admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:38:46.022860 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1343/2698] 10.222.0.49 () {58 vars in 1121 bytes} [Wed Jun 15 10:38:45 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 10:38:46.023757 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7242bf-abe9-424a-b30d-5b326482f4dc admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:38:46.046972 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7242bf-abe9-424a-b30d-5b326482f4dc admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:38:46.054965 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d4bfb5-1dce-434c-bfba-d6f05f326a42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:38:46.057714 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d4bfb5-1dce-434c-bfba-d6f05f326a42 admin admin] REQUEST_METHOD: `PUT` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:38:46.058218 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d4bfb5-1dce-434c-bfba-d6f05f326a42 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:38:46.058707 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d4bfb5-1dce-434c-bfba-d6f05f326a42 admin admin] PATH_INFO: `/v3/projects/e6b481bd220347a7a8843e470264615d/users/ebadfb187c4e482eba2255bf19649655/roles/6876e57cc51c448097ab109d35b2a3da` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:38:46.081517 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca00244-4c56-4ed1-8405-1e4da1d16477 admin admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:38:46.084294 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d4bfb5-1dce-434c-bfba-d6f05f326a42 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e6b481bd220347a7a8843e470264615d, user_id=ebadfb187c4e482eba2255bf19649655, role_id=6876e57cc51c448097ab109d35b2a3da)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:38:46.085781 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9572ff6e-3873-474e-a191-def7170da9c9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20d1327553404b80aecee87c78fcb52b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '559b3e713f5844b6ae37151d56dd419f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:38:46.086452 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9572ff6e-3873-474e-a191-def7170da9c9 service cinder] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:38:46.086730 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9572ff6e-3873-474e-a191-def7170da9c9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:38:46.087006 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9572ff6e-3873-474e-a191-def7170da9c9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:38:46.089069 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d4bfb5-1dce-434c-bfba-d6f05f326a42 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:38:46.110566 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9572ff6e-3873-474e-a191-def7170da9c9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:38:46.113382 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9572ff6e-3873-474e-a191-def7170da9c9 service cinder] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:38:46.120723 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1344/2699] 10.222.0.49 () {60 vars in 1482 bytes} [Wed Jun 15 10:38:46 2022] PUT /identity/v3/projects/e6b481bd220347a7a8843e470264615d/users/ebadfb187c4e482eba2255bf19649655/roles/6876e57cc51c448097ab109d35b2a3da => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:38:46.131125 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1356/2700] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Jun 15 10:38:45 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:38:46.131125 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9572ff6e-3873-474e-a191-def7170da9c9 service cinder] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:38:46.136292 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea54b133-4c6c-455f-9b9f-d8718551efa9 None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:38:46.136838 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea54b133-4c6c-455f-9b9f-d8718551efa9 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:38:46.137356 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea54b133-4c6c-455f-9b9f-d8718551efa9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:38:46.137767 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9572ff6e-3873-474e-a191-def7170da9c9 service cinder] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:38:46.182171 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-ea54b133-4c6c-455f-9b9f-d8718551efa9 None None] MFA Rules not processed for user `ebadfb187c4e482eba2255bf19649655`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:38:46.184604 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-ea54b133-4c6c-455f-9b9f-d8718551efa9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:38:46.235594 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297a2daa-00fd-4d31-825f-cda634407967 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:38:46.236149 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297a2daa-00fd-4d31-825f-cda634407967 admin admin] REQUEST_METHOD: `PUT` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:38:46.236430 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297a2daa-00fd-4d31-825f-cda634407967 admin admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:38:46.236707 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297a2daa-00fd-4d31-825f-cda634407967 admin admin] PATH_INFO: `/v3/projects/c0896c3d67384b1493e5e6287b8ae9e7/users/8714c96315fe434584ccdaf2893b8ad4/roles/6876e57cc51c448097ab109d35b2a3da` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:38:46.267683 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-297a2daa-00fd-4d31-825f-cda634407967 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c0896c3d67384b1493e5e6287b8ae9e7, user_id=8714c96315fe434584ccdaf2893b8ad4, role_id=6876e57cc51c448097ab109d35b2a3da)` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:38:46.285099 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-297a2daa-00fd-4d31-825f-cda634407967 admin admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:38:46.317372 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1357/2701] 10.222.0.49 () {60 vars in 1482 bytes} [Wed Jun 15 10:38:46 2022] PUT /identity/v3/projects/c0896c3d67384b1493e5e6287b8ae9e7/users/8714c96315fe434584ccdaf2893b8ad4/roles/6876e57cc51c448097ab109d35b2a3da => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:38:46.330414 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764a32ce-0f99-444b-bcc5-3a883900c83c None None] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:38:46.330700 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764a32ce-0f99-444b-bcc5-3a883900c83c None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:38:46.330981 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764a32ce-0f99-444b-bcc5-3a883900c83c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:38:46.366905 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1345/2702] 10.222.0.49 () {58 vars in 957 bytes} [Wed Jun 15 10:38:46 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 10:38:46.380519 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea54b133-4c6c-455f-9b9f-d8718551efa9 None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:38:46.390924 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.auth.core [None req-764a32ce-0f99-444b-bcc5-3a883900c83c None None] MFA Rules not processed for user `8714c96315fe434584ccdaf2893b8ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:38:46.394005 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:38:46.395163 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-764a32ce-0f99-444b-bcc5-3a883900c83c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:38:46.473110 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea54b133-4c6c-455f-9b9f-d8718551efa9 None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:38:46.499058 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:38:46.511789 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1358/2703] 10.222.0.49 () {58 vars in 957 bytes} [Wed Jun 15 10:38:46 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:38:46.512911 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764a32ce-0f99-444b-bcc5-3a883900c83c None None] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:38:46.596833 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2e5104-7b88-40b3-8ebc-d6d1631cf921 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20d1327553404b80aecee87c78fcb52b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '559b3e713f5844b6ae37151d56dd419f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:38:46.597690 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2e5104-7b88-40b3-8ebc-d6d1631cf921 service cinder] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:38:46.597894 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2e5104-7b88-40b3-8ebc-d6d1631cf921 service cinder] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:38:46.598161 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2e5104-7b88-40b3-8ebc-d6d1631cf921 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:38:46.622489 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764a32ce-0f99-444b-bcc5-3a883900c83c None None] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:38:46.633730 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ccd494d-17e8-4478-a7ba-0d28e38b0e9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:38:46.634524 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccd494d-17e8-4478-a7ba-0d28e38b0e9c admin admin] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:38:46.635107 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccd494d-17e8-4478-a7ba-0d28e38b0e9c admin admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:38:46.635617 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccd494d-17e8-4478-a7ba-0d28e38b0e9c admin admin] PATH_INFO: `/v3/projects` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:38:46.636551 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ccd494d-17e8-4478-a7ba-0d28e38b0e9c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:38:46.639767 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ccd494d-17e8-4478-a7ba-0d28e38b0e9c admin admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:38:46.709950 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff2e5104-7b88-40b3-8ebc-d6d1631cf921 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:38:46.712783 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff2e5104-7b88-40b3-8ebc-d6d1631cf921 service cinder] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:38:46.713312 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1359/2704] 10.222.0.49 () {60 vars in 1152 bytes} [Wed Jun 15 10:38:46 2022] POST /identity/v3/projects => generated 373 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 10:38:46.717542 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ccd494d-17e8-4478-a7ba-0d28e38b0e9c admin admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:38:46.733952 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1346/2705] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Jun 15 10:38:46 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:38:46.735815 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2e5104-7b88-40b3-8ebc-d6d1631cf921 service cinder] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:38:46.751399 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2e5104-7b88-40b3-8ebc-d6d1631cf921 service cinder] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:38:46.835702 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4e23be-566a-456f-8a56-f061f15ba2a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:38:46.836682 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4e23be-566a-456f-8a56-f061f15ba2a2 admin admin] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:38:46.837584 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4e23be-566a-456f-8a56-f061f15ba2a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:38:46.837932 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4e23be-566a-456f-8a56-f061f15ba2a2 admin admin] PATH_INFO: `/v3/users` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:38:46.838747 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d4e23be-566a-456f-8a56-f061f15ba2a2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:38:46.840904 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d4e23be-566a-456f-8a56-f061f15ba2a2 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:38:46.867429 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1347/2706] 10.222.0.49 () {60 vars in 1143 bytes} [Wed Jun 15 10:38:46 2022] POST /identity/v3/users => generated 293 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 10:38:46.871572 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ccd494d-17e8-4478-a7ba-0d28e38b0e9c admin admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:38:46.871734 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4e23be-566a-456f-8a56-f061f15ba2a2 admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:38:46.877965 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4662ec75-3f05-4aa6-a618-87b9fa383465 tempest-VolumeQuotaClassesTest-1006105901 tempest-VolumeQuotaClassesTest-1006105901-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36dc549976e940f1b42b30580eafde65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0fd3f208877a490baf55581ebbc5a701', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:38:46.879105 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4662ec75-3f05-4aa6-a618-87b9fa383465 tempest-VolumeQuotaClassesTest-1006105901 tempest-VolumeQuotaClassesTest-1006105901-project-admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:38:46.879552 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4662ec75-3f05-4aa6-a618-87b9fa383465 tempest-VolumeQuotaClassesTest-1006105901 tempest-VolumeQuotaClassesTest-1006105901-project-admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:38:46.879937 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4662ec75-3f05-4aa6-a618-87b9fa383465 tempest-VolumeQuotaClassesTest-1006105901 tempest-VolumeQuotaClassesTest-1006105901-project-admin] PATH_INFO: `/v3/domains` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:38:46.880783 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4662ec75-3f05-4aa6-a618-87b9fa383465 tempest-VolumeQuotaClassesTest-1006105901 tempest-VolumeQuotaClassesTest-1006105901-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:38:46.881346 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4662ec75-3f05-4aa6-a618-87b9fa383465 tempest-VolumeQuotaClassesTest-1006105901 tempest-VolumeQuotaClassesTest-1006105901-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:38:46.883073 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4662ec75-3f05-4aa6-a618-87b9fa383465 tempest-VolumeQuotaClassesTest-1006105901 tempest-VolumeQuotaClassesTest-1006105901-project-admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:38:46.890986 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4e23be-566a-456f-8a56-f061f15ba2a2 admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:38:46.899435 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1360/2707] 10.222.0.49 () {58 vars in 1152 bytes} [Wed Jun 15 10:38:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:38:46.901852 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681b29e6-0fac-48d3-a6e6-ce7c1535bea3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:38:46.902630 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681b29e6-0fac-48d3-a6e6-ce7c1535bea3 admin admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:38:46.902940 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681b29e6-0fac-48d3-a6e6-ce7c1535bea3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:38:46.903275 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681b29e6-0fac-48d3-a6e6-ce7c1535bea3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:38:46.904085 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681b29e6-0fac-48d3-a6e6-ce7c1535bea3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:38:46.905783 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681b29e6-0fac-48d3-a6e6-ce7c1535bea3 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:38:46.907789 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:38:46.908584 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4662ec75-3f05-4aa6-a618-87b9fa383465 tempest-VolumeQuotaClassesTest-1006105901 tempest-VolumeQuotaClassesTest-1006105901-project-admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:38:46.929117 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4662ec75-3f05-4aa6-a618-87b9fa383465 tempest-VolumeQuotaClassesTest-1006105901 tempest-VolumeQuotaClassesTest-1006105901-project-admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:38:46.933262 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1348/2708] 10.222.0.49 () {58 vars in 1121 bytes} [Wed Jun 15 10:38:46 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 15 10:38:46.936081 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681b29e6-0fac-48d3-a6e6-ce7c1535bea3 admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:38:46.943715 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2fcef12-3841-4134-972d-6c03906d72ec tempest-VolumeQuotaClassesTest-1006105901 tempest-VolumeQuotaClassesTest-1006105901-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36dc549976e940f1b42b30580eafde65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0fd3f208877a490baf55581ebbc5a701', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:38:46.944536 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fcef12-3841-4134-972d-6c03906d72ec tempest-VolumeQuotaClassesTest-1006105901 tempest-VolumeQuotaClassesTest-1006105901-project-admin] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:38:46.944842 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fcef12-3841-4134-972d-6c03906d72ec tempest-VolumeQuotaClassesTest-1006105901 tempest-VolumeQuotaClassesTest-1006105901-project-admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:38:46.945126 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fcef12-3841-4134-972d-6c03906d72ec tempest-VolumeQuotaClassesTest-1006105901 tempest-VolumeQuotaClassesTest-1006105901-project-admin] PATH_INFO: `/v3/projects` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:38:46.945825 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2fcef12-3841-4134-972d-6c03906d72ec tempest-VolumeQuotaClassesTest-1006105901 tempest-VolumeQuotaClassesTest-1006105901-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:38:46.947799 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2fcef12-3841-4134-972d-6c03906d72ec tempest-VolumeQuotaClassesTest-1006105901 tempest-VolumeQuotaClassesTest-1006105901-project-admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:38:46.956357 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681b29e6-0fac-48d3-a6e6-ce7c1535bea3 admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:38:46.962429 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89252747-7901-469d-8496-0191b7d124df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:38:46.963561 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89252747-7901-469d-8496-0191b7d124df admin admin] REQUEST_METHOD: `PUT` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:38:46.963836 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89252747-7901-469d-8496-0191b7d124df admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:38:46.964115 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89252747-7901-469d-8496-0191b7d124df admin admin] PATH_INFO: `/v3/projects/477acc8404da4fad80d87bbc7e36c497/users/9771429b486c4bce9d48dd246273d031/roles/8051174e7a7947d5b991ebd793c2e543` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:38:46.981738 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89252747-7901-469d-8496-0191b7d124df admin admin] RBAC: Authorizing `identity:create_grant(project_id=477acc8404da4fad80d87bbc7e36c497, user_id=9771429b486c4bce9d48dd246273d031, role_id=8051174e7a7947d5b991ebd793c2e543)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:38:46.985463 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89252747-7901-469d-8496-0191b7d124df admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:38:47.009782 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1349/2709] 10.222.0.49 () {60 vars in 1482 bytes} [Wed Jun 15 10:38:46 2022] PUT /identity/v3/projects/477acc8404da4fad80d87bbc7e36c497/users/9771429b486c4bce9d48dd246273d031/roles/8051174e7a7947d5b991ebd793c2e543 => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:38:47.013804 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89252747-7901-469d-8496-0191b7d124df admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:38:47.021781 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1361/2710] 10.222.0.49 () {60 vars in 1152 bytes} [Wed Jun 15 10:38:46 2022] POST /identity/v3/projects => generated 353 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 10:38:47.024559 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2fcef12-3841-4134-972d-6c03906d72ec tempest-VolumeQuotaClassesTest-1006105901 tempest-VolumeQuotaClassesTest-1006105901-project-admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:38:47.151733 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89252747-7901-469d-8496-0191b7d124df admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:38:47.157122 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:38:47.166503 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2fcef12-3841-4134-972d-6c03906d72ec tempest-VolumeQuotaClassesTest-1006105901 tempest-VolumeQuotaClassesTest-1006105901-project-admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:38:47.178292 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904ae7fe-152d-42e9-ba98-1ad2390696c5 tempest-VolumeQuotaClassesTest-1006105901 tempest-VolumeQuotaClassesTest-1006105901-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36dc549976e940f1b42b30580eafde65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0fd3f208877a490baf55581ebbc5a701', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:38:47.179127 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904ae7fe-152d-42e9-ba98-1ad2390696c5 tempest-VolumeQuotaClassesTest-1006105901 tempest-VolumeQuotaClassesTest-1006105901-project-admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:38:47.180618 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904ae7fe-152d-42e9-ba98-1ad2390696c5 tempest-VolumeQuotaClassesTest-1006105901 tempest-VolumeQuotaClassesTest-1006105901-project-admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:38:47.180977 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904ae7fe-152d-42e9-ba98-1ad2390696c5 tempest-VolumeQuotaClassesTest-1006105901 tempest-VolumeQuotaClassesTest-1006105901-project-admin] PATH_INFO: `/v3/domains` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:38:47.182297 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904ae7fe-152d-42e9-ba98-1ad2390696c5 tempest-VolumeQuotaClassesTest-1006105901 tempest-VolumeQuotaClassesTest-1006105901-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:38:47.183619 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904ae7fe-152d-42e9-ba98-1ad2390696c5 tempest-VolumeQuotaClassesTest-1006105901 tempest-VolumeQuotaClassesTest-1006105901-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:38:47.185349 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904ae7fe-152d-42e9-ba98-1ad2390696c5 tempest-VolumeQuotaClassesTest-1006105901 tempest-VolumeQuotaClassesTest-1006105901-project-admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:38:47.242720 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1362/2711] 10.222.0.49 () {58 vars in 1152 bytes} [Wed Jun 15 10:38:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:38:47.297909 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575a371c-b0e6-4022-b019-eb18fa5d586d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:38:47.298556 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575a371c-b0e6-4022-b019-eb18fa5d586d admin admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:38:47.298895 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575a371c-b0e6-4022-b019-eb18fa5d586d admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:38:47.299322 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575a371c-b0e6-4022-b019-eb18fa5d586d admin admin] PATH_INFO: `/v3/roles` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:38:47.300271 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-575a371c-b0e6-4022-b019-eb18fa5d586d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:38:47.301993 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-575a371c-b0e6-4022-b019-eb18fa5d586d admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:38:47.318097 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904ae7fe-152d-42e9-ba98-1ad2390696c5 tempest-VolumeQuotaClassesTest-1006105901 tempest-VolumeQuotaClassesTest-1006105901-project-admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:38:47.324129 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1350/2712] 10.222.0.49 () {58 vars in 1121 bytes} [Wed Jun 15 10:38:47 2022] GET /identity/v3/roles => generated 1420 bytes in 310 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 10:38:47.328220 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575a371c-b0e6-4022-b019-eb18fa5d586d admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:38:47.334856 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904ae7fe-152d-42e9-ba98-1ad2390696c5 tempest-VolumeQuotaClassesTest-1006105901 tempest-VolumeQuotaClassesTest-1006105901-project-admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:38:47.352480 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92b98df-db39-4534-9f22-efc303bbffca tempest-VolumeQuotaClassesTest-1006105901 tempest-VolumeQuotaClassesTest-1006105901-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36dc549976e940f1b42b30580eafde65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0fd3f208877a490baf55581ebbc5a701', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:38:47.353388 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92b98df-db39-4534-9f22-efc303bbffca tempest-VolumeQuotaClassesTest-1006105901 tempest-VolumeQuotaClassesTest-1006105901-project-admin] REQUEST_METHOD: `DELETE` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:38:47.354607 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92b98df-db39-4534-9f22-efc303bbffca tempest-VolumeQuotaClassesTest-1006105901 tempest-VolumeQuotaClassesTest-1006105901-project-admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:38:47.354903 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92b98df-db39-4534-9f22-efc303bbffca tempest-VolumeQuotaClassesTest-1006105901 tempest-VolumeQuotaClassesTest-1006105901-project-admin] PATH_INFO: `/v3/projects/4373308af87d4b4baae19684ffdf7ede` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:38:47.357170 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575a371c-b0e6-4022-b019-eb18fa5d586d admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:38:47.360300 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b92b98df-db39-4534-9f22-efc303bbffca tempest-VolumeQuotaClassesTest-1006105901 tempest-VolumeQuotaClassesTest-1006105901-project-admin] RBAC: Authorizing `identity:delete_project(project_id=4373308af87d4b4baae19684ffdf7ede)` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:38:47.362263 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b92b98df-db39-4534-9f22-efc303bbffca tempest-VolumeQuotaClassesTest-1006105901 tempest-VolumeQuotaClassesTest-1006105901-project-admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:38:47.373453 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8181efd8-5a58-48ae-98ea-94d14c87d419 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:38:47.374382 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8181efd8-5a58-48ae-98ea-94d14c87d419 admin admin] REQUEST_METHOD: `PUT` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:38:47.374667 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8181efd8-5a58-48ae-98ea-94d14c87d419 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:38:47.374947 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8181efd8-5a58-48ae-98ea-94d14c87d419 admin admin] PATH_INFO: `/v3/projects/477acc8404da4fad80d87bbc7e36c497/users/9771429b486c4bce9d48dd246273d031/roles/8051174e7a7947d5b991ebd793c2e543` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:38:47.378735 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8181efd8-5a58-48ae-98ea-94d14c87d419 admin admin] RBAC: Authorizing `identity:create_grant(project_id=477acc8404da4fad80d87bbc7e36c497, user_id=9771429b486c4bce9d48dd246273d031, role_id=8051174e7a7947d5b991ebd793c2e543)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:38:47.381942 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8181efd8-5a58-48ae-98ea-94d14c87d419 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:38:47.415688 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1351/2713] 10.222.0.49 () {60 vars in 1482 bytes} [Wed Jun 15 10:38:47 2022] PUT /identity/v3/projects/477acc8404da4fad80d87bbc7e36c497/users/9771429b486c4bce9d48dd246273d031/roles/8051174e7a7947d5b991ebd793c2e543 => generated 0 bytes in 91 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:38:47.426716 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0131780-7c56-4eb9-b24a-d45b1d2dbb91 None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:38:47.427395 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0131780-7c56-4eb9-b24a-d45b1d2dbb91 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:38:47.427777 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0131780-7c56-4eb9-b24a-d45b1d2dbb91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:38:47.473099 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-b92b98df-db39-4534-9f22-efc303bbffca tempest-VolumeQuotaClassesTest-1006105901 tempest-VolumeQuotaClassesTest-1006105901-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4373308af87d4b4baae19684ffdf7ede', 'request_id': 'req-b92b98df-db39-4534-9f22-efc303bbffca'} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:38:47.476760 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-f0131780-7c56-4eb9-b24a-d45b1d2dbb91 None None] MFA Rules not processed for user `9771429b486c4bce9d48dd246273d031`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:38:47.478113 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-f0131780-7c56-4eb9-b24a-d45b1d2dbb91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:38:47.489012 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-b92b98df-db39-4534-9f22-efc303bbffca tempest-VolumeQuotaClassesTest-1006105901 tempest-VolumeQuotaClassesTest-1006105901-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4373308af87d4b4baae19684ffdf7ede', 'request_id': 'req-b92b98df-db39-4534-9f22-efc303bbffca'} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:38:47.499717 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-b92b98df-db39-4534-9f22-efc303bbffca tempest-VolumeQuotaClassesTest-1006105901 tempest-VolumeQuotaClassesTest-1006105901-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4373308af87d4b4baae19684ffdf7ede', 'request_id': 'req-b92b98df-db39-4534-9f22-efc303bbffca'} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:38:47.543377 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-b92b98df-db39-4534-9f22-efc303bbffca tempest-VolumeQuotaClassesTest-1006105901 tempest-VolumeQuotaClassesTest-1006105901-project-admin] The token cache is being invalidate because project 4373308af87d4b4baae19684ffdf7ede was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80119) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 10:38:47.543962 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-b92b98df-db39-4534-9f22-efc303bbffca tempest-VolumeQuotaClassesTest-1006105901 tempest-VolumeQuotaClassesTest-1006105901-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:38:47.546394 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-b92b98df-db39-4534-9f22-efc303bbffca tempest-VolumeQuotaClassesTest-1006105901 tempest-VolumeQuotaClassesTest-1006105901-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:38:47.555488 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1363/2714] 10.222.0.49 () {58 vars in 1232 bytes} [Wed Jun 15 10:38:47 2022] DELETE /identity/v3/projects/4373308af87d4b4baae19684ffdf7ede => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 10:38:47.590815 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:38:47.627991 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1352/2715] 10.222.0.49 () {58 vars in 957 bytes} [Wed Jun 15 10:38:47 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 10:38:47.649270 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92b98df-db39-4534-9f22-efc303bbffca tempest-VolumeQuotaClassesTest-1006105901 tempest-VolumeQuotaClassesTest-1006105901-project-admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:38:47.650747 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-b92b98df-db39-4534-9f22-efc303bbffca tempest-VolumeQuotaClassesTest-1006105901 tempest-VolumeQuotaClassesTest-1006105901-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:38:47.818221 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92b98df-db39-4534-9f22-efc303bbffca tempest-VolumeQuotaClassesTest-1006105901 tempest-VolumeQuotaClassesTest-1006105901-project-admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:38:47.826253 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc97db9-1388-4026-9827-f99895d6c761 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20d1327553404b80aecee87c78fcb52b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '559b3e713f5844b6ae37151d56dd419f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:38:47.826948 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc97db9-1388-4026-9827-f99895d6c761 service cinder] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:38:47.830088 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc97db9-1388-4026-9827-f99895d6c761 service cinder] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:38:47.830444 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc97db9-1388-4026-9827-f99895d6c761 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:38:47.835654 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-1dc97db9-1388-4026-9827-f99895d6c761 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:38:47.995130 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0131780-7c56-4eb9-b24a-d45b1d2dbb91 None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:38:47.996659 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-f0131780-7c56-4eb9-b24a-d45b1d2dbb91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:38:48.033619 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc97db9-1388-4026-9827-f99895d6c761 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:38:48.049721 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc97db9-1388-4026-9827-f99895d6c761 service cinder] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:38:48.096745 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1364/2716] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Jun 15 10:38:47 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 449 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:38:48.176196 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0131780-7c56-4eb9-b24a-d45b1d2dbb91 None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:38:48.190449 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fefd653-e096-4a30-b1e0-23f138d8ab6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:38:48.191409 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fefd653-e096-4a30-b1e0-23f138d8ab6b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:38:48.191696 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fefd653-e096-4a30-b1e0-23f138d8ab6b admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:38:48.191971 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fefd653-e096-4a30-b1e0-23f138d8ab6b admin admin] PATH_INFO: `/v3/users/666c90cfe3ce4e53ad4ab1dcfd2be37d` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:38:48.193945 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fefd653-e096-4a30-b1e0-23f138d8ab6b admin admin] RBAC: Authorizing `identity:delete_user(user_id=666c90cfe3ce4e53ad4ab1dcfd2be37d)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:38:48.210993 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fefd653-e096-4a30-b1e0-23f138d8ab6b admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:38:48.363889 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-8fefd653-e096-4a30-b1e0-23f138d8ab6b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '666c90cfe3ce4e53ad4ab1dcfd2be37d'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:38:48.388974 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-8fefd653-e096-4a30-b1e0-23f138d8ab6b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '666c90cfe3ce4e53ad4ab1dcfd2be37d'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:38:48.389898 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-8fefd653-e096-4a30-b1e0-23f138d8ab6b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '666c90cfe3ce4e53ad4ab1dcfd2be37d'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:38:48.393904 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-8fefd653-e096-4a30-b1e0-23f138d8ab6b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '666c90cfe3ce4e53ad4ab1dcfd2be37d'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:38:48.414980 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-8fefd653-e096-4a30-b1e0-23f138d8ab6b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '666c90cfe3ce4e53ad4ab1dcfd2be37d'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:38:48.482422 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1353/2717] 10.222.0.49 () {58 vars in 1223 bytes} [Wed Jun 15 10:38:47 2022] DELETE /identity/v3/users/666c90cfe3ce4e53ad4ab1dcfd2be37d => generated 0 bytes in 485 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 10:38:48.484148 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3754715b-c03f-4740-ba16-c111a5dbb628 None None] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:38:48.484819 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3754715b-c03f-4740-ba16-c111a5dbb628 None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:38:48.484819 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3754715b-c03f-4740-ba16-c111a5dbb628 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:38:48.518754 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.auth.core [None req-3754715b-c03f-4740-ba16-c111a5dbb628 None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:38:48.520167 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-3754715b-c03f-4740-ba16-c111a5dbb628 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:38:48.594698 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:38:48.683592 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1365/2718] 10.222.0.49 () {58 vars in 957 bytes} [Wed Jun 15 10:38:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:38:48.695665 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fefd653-e096-4a30-b1e0-23f138d8ab6b admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:38:48.697654 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-8fefd653-e096-4a30-b1e0-23f138d8ab6b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:38:48.849500 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fefd653-e096-4a30-b1e0-23f138d8ab6b admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:38:48.855421 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c28906-6e84-4a25-81bb-2e7e5068ae5b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0cb44a17d205431793e9a8791e477a32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '559b3e713f5844b6ae37151d56dd419f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:38:48.856239 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c28906-6e84-4a25-81bb-2e7e5068ae5b service neutron] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:38:48.856533 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c28906-6e84-4a25-81bb-2e7e5068ae5b service neutron] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:38:48.856815 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c28906-6e84-4a25-81bb-2e7e5068ae5b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:38:48.869371 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c28906-6e84-4a25-81bb-2e7e5068ae5b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:38:48.871399 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c28906-6e84-4a25-81bb-2e7e5068ae5b service neutron] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:38:48.889888 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1354/2719] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Jun 15 10:38:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 195 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:38:49.359361 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3754715b-c03f-4740-ba16-c111a5dbb628 None None] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:38:49.361442 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-3754715b-c03f-4740-ba16-c111a5dbb628 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:38:49.379793 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3754715b-c03f-4740-ba16-c111a5dbb628 None None] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:38:49.385766 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de118e9d-401b-42c2-be40-344f79354553 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:38:49.386806 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de118e9d-401b-42c2-be40-344f79354553 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:38:49.387209 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de118e9d-401b-42c2-be40-344f79354553 admin admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:38:49.387589 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de118e9d-401b-42c2-be40-344f79354553 admin admin] PATH_INFO: `/v3/projects/4a0f117ef57a4c08bdf37604b76a6793` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:38:49.389671 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de118e9d-401b-42c2-be40-344f79354553 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4a0f117ef57a4c08bdf37604b76a6793)` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:38:49.392639 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de118e9d-401b-42c2-be40-344f79354553 admin admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:38:49.508608 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-de118e9d-401b-42c2-be40-344f79354553 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4a0f117ef57a4c08bdf37604b76a6793', 'request_id': 'req-de118e9d-401b-42c2-be40-344f79354553'} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:38:49.527925 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-de118e9d-401b-42c2-be40-344f79354553 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4a0f117ef57a4c08bdf37604b76a6793', 'request_id': 'req-de118e9d-401b-42c2-be40-344f79354553'} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:38:49.540544 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-de118e9d-401b-42c2-be40-344f79354553 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4a0f117ef57a4c08bdf37604b76a6793', 'request_id': 'req-de118e9d-401b-42c2-be40-344f79354553'} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:38:49.567788 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-de118e9d-401b-42c2-be40-344f79354553 admin admin] The token cache is being invalidate because project 4a0f117ef57a4c08bdf37604b76a6793 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80119) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 10:38:49.568399 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-de118e9d-401b-42c2-be40-344f79354553 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:38:49.569314 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-de118e9d-401b-42c2-be40-344f79354553 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:38:49.571429 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1366/2720] 10.222.0.49 () {58 vars in 1232 bytes} [Wed Jun 15 10:38:49 2022] DELETE /identity/v3/projects/4a0f117ef57a4c08bdf37604b76a6793 => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:38:49.577605 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c28906-6e84-4a25-81bb-2e7e5068ae5b service neutron] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:38:49.617478 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c28906-6e84-4a25-81bb-2e7e5068ae5b service neutron] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:38:49.627525 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:38:49.743313 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4919504c-482d-4259-9d8a-2d7bde6328ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:38:49.744525 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4919504c-482d-4259-9d8a-2d7bde6328ff admin admin] REQUEST_METHOD: `DELETE` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:38:49.745053 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4919504c-482d-4259-9d8a-2d7bde6328ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:38:49.745712 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4919504c-482d-4259-9d8a-2d7bde6328ff admin admin] PATH_INFO: `/v3/users/36dc549976e940f1b42b30580eafde65` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:38:49.748011 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4919504c-482d-4259-9d8a-2d7bde6328ff admin admin] RBAC: Authorizing `identity:delete_user(user_id=36dc549976e940f1b42b30580eafde65)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:38:49.752567 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4919504c-482d-4259-9d8a-2d7bde6328ff admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:38:49.868310 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-4919504c-482d-4259-9d8a-2d7bde6328ff admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '36dc549976e940f1b42b30580eafde65'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:38:49.880732 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-4919504c-482d-4259-9d8a-2d7bde6328ff admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '36dc549976e940f1b42b30580eafde65'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:38:49.881501 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-4919504c-482d-4259-9d8a-2d7bde6328ff admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '36dc549976e940f1b42b30580eafde65'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:38:49.882232 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-4919504c-482d-4259-9d8a-2d7bde6328ff admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '36dc549976e940f1b42b30580eafde65'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:38:49.894419 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-4919504c-482d-4259-9d8a-2d7bde6328ff admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '36dc549976e940f1b42b30580eafde65'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:38:49.917567 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1355/2721] 10.222.0.49 () {58 vars in 1223 bytes} [Wed Jun 15 10:38:49 2022] DELETE /identity/v3/users/36dc549976e940f1b42b30580eafde65 => generated 0 bytes in 342 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:38:50.313539 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de118e9d-401b-42c2-be40-344f79354553 admin admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:38:50.317219 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-de118e9d-401b-42c2-be40-344f79354553 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:38:50.340768 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de118e9d-401b-42c2-be40-344f79354553 admin admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:38:50.451812 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656bbbec-63ba-49f1-8148-d16733bf8abf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:38:50.452348 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656bbbec-63ba-49f1-8148-d16733bf8abf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:38:50.452628 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656bbbec-63ba-49f1-8148-d16733bf8abf admin admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:38:50.452904 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656bbbec-63ba-49f1-8148-d16733bf8abf admin admin] PATH_INFO: `/v3/projects/0fd3f208877a490baf55581ebbc5a701` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:38:50.455024 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-656bbbec-63ba-49f1-8148-d16733bf8abf admin admin] RBAC: Authorizing `identity:delete_project(project_id=0fd3f208877a490baf55581ebbc5a701)` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:38:50.457086 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-656bbbec-63ba-49f1-8148-d16733bf8abf admin admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:38:50.521884 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-656bbbec-63ba-49f1-8148-d16733bf8abf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0fd3f208877a490baf55581ebbc5a701', 'request_id': 'req-656bbbec-63ba-49f1-8148-d16733bf8abf'} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:38:50.528429 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:38:50.536417 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-656bbbec-63ba-49f1-8148-d16733bf8abf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0fd3f208877a490baf55581ebbc5a701', 'request_id': 'req-656bbbec-63ba-49f1-8148-d16733bf8abf'} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:38:50.543376 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-656bbbec-63ba-49f1-8148-d16733bf8abf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0fd3f208877a490baf55581ebbc5a701', 'request_id': 'req-656bbbec-63ba-49f1-8148-d16733bf8abf'} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:38:50.557722 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-656bbbec-63ba-49f1-8148-d16733bf8abf admin admin] The token cache is being invalidate because project 0fd3f208877a490baf55581ebbc5a701 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80119) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 10:38:50.558101 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-656bbbec-63ba-49f1-8148-d16733bf8abf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:38:50.558802 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-656bbbec-63ba-49f1-8148-d16733bf8abf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:38:50.566036 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1367/2722] 10.222.0.49 () {58 vars in 1232 bytes} [Wed Jun 15 10:38:50 2022] DELETE /identity/v3/projects/0fd3f208877a490baf55581ebbc5a701 => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:38:50.567276 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f659de13-6f0b-4bdc-9427-47ab7fe2585b None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:38:50.567645 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f659de13-6f0b-4bdc-9427-47ab7fe2585b None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:38:50.567862 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f659de13-6f0b-4bdc-9427-47ab7fe2585b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:38:50.595808 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-f659de13-6f0b-4bdc-9427-47ab7fe2585b None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:38:50.599750 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-f659de13-6f0b-4bdc-9427-47ab7fe2585b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:38:50.600055 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656bbbec-63ba-49f1-8148-d16733bf8abf admin admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:38:50.601456 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-656bbbec-63ba-49f1-8148-d16733bf8abf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:38:50.721677 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656bbbec-63ba-49f1-8148-d16733bf8abf admin admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:38:50.741066 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe7acc6-b084-4256-a79b-8e918e1e54a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:38:50.741066 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe7acc6-b084-4256-a79b-8e918e1e54a7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:38:50.741066 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe7acc6-b084-4256-a79b-8e918e1e54a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:38:50.741066 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe7acc6-b084-4256-a79b-8e918e1e54a7 admin admin] PATH_INFO: `/v3/users/ebadfb187c4e482eba2255bf19649655` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:38:50.742111 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1356/2723] 10.222.0.49 () {58 vars in 957 bytes} [Wed Jun 15 10:38:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:38:50.743068 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe7acc6-b084-4256-a79b-8e918e1e54a7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ebadfb187c4e482eba2255bf19649655)` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:38:50.744117 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd7f2e7-28c4-469e-ab05-1a1c1f338110 None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:38:50.744617 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd7f2e7-28c4-469e-ab05-1a1c1f338110 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:38:50.745093 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd7f2e7-28c4-469e-ab05-1a1c1f338110 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:38:50.745711 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe7acc6-b084-4256-a79b-8e918e1e54a7 admin admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:38:50.782864 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-fdd7f2e7-28c4-469e-ab05-1a1c1f338110 None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:38:50.784567 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-fdd7f2e7-28c4-469e-ab05-1a1c1f338110 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:38:50.809207 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1357/2724] 10.222.0.49 () {58 vars in 957 bytes} [Wed Jun 15 10:38:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 66 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:38:50.814321 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd7f2e7-28c4-469e-ab05-1a1c1f338110 None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:38:50.828473 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd7f2e7-28c4-469e-ab05-1a1c1f338110 None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:38:50.836796 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fb7b0e-354b-4a2f-a96b-6afe29d58939 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:38:50.837676 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fb7b0e-354b-4a2f-a96b-6afe29d58939 admin admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:38:50.838109 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fb7b0e-354b-4a2f-a96b-6afe29d58939 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:38:50.839073 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fb7b0e-354b-4a2f-a96b-6afe29d58939 admin admin] PATH_INFO: `/v3/domains` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:38:50.840761 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91fb7b0e-354b-4a2f-a96b-6afe29d58939 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:38:50.841336 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91fb7b0e-354b-4a2f-a96b-6afe29d58939 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:38:50.844927 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91fb7b0e-354b-4a2f-a96b-6afe29d58939 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:38:50.854942 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-8fe7acc6-b084-4256-a79b-8e918e1e54a7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ebadfb187c4e482eba2255bf19649655'} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:38:50.867032 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-8fe7acc6-b084-4256-a79b-8e918e1e54a7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ebadfb187c4e482eba2255bf19649655'} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:38:50.870851 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1358/2725] 10.222.0.49 () {58 vars in 1152 bytes} [Wed Jun 15 10:38:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:38:50.871552 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-8fe7acc6-b084-4256-a79b-8e918e1e54a7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ebadfb187c4e482eba2255bf19649655'} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:38:50.872055 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fb7b0e-354b-4a2f-a96b-6afe29d58939 admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:38:50.872535 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-8fe7acc6-b084-4256-a79b-8e918e1e54a7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ebadfb187c4e482eba2255bf19649655'} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:38:50.875238 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-91fb7b0e-354b-4a2f-a96b-6afe29d58939 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:38:50.887886 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-8fe7acc6-b084-4256-a79b-8e918e1e54a7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ebadfb187c4e482eba2255bf19649655'} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:38:50.896766 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fb7b0e-354b-4a2f-a96b-6afe29d58939 admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:38:50.902975 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb040f07-fa38-4592-8510-d40ff0fdebbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:38:50.903865 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb040f07-fa38-4592-8510-d40ff0fdebbe admin admin] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:38:50.904253 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb040f07-fa38-4592-8510-d40ff0fdebbe admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:38:50.904771 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb040f07-fa38-4592-8510-d40ff0fdebbe admin admin] PATH_INFO: `/v3/projects` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:38:50.905588 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb040f07-fa38-4592-8510-d40ff0fdebbe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:38:50.908359 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb040f07-fa38-4592-8510-d40ff0fdebbe admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:38:50.914171 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:38:50.940879 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1368/2726] 10.222.0.49 () {58 vars in 1223 bytes} [Wed Jun 15 10:38:50 2022] DELETE /identity/v3/users/ebadfb187c4e482eba2255bf19649655 => generated 0 bytes in 338 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 10:38:50.942755 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3427fe3-d9e7-476d-9d06-6a7c0a731b03 None None] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:38:50.943057 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3427fe3-d9e7-476d-9d06-6a7c0a731b03 None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:38:50.943380 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3427fe3-d9e7-476d-9d06-6a7c0a731b03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:38:50.962614 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1359/2727] 10.222.0.49 () {60 vars in 1152 bytes} [Wed Jun 15 10:38:50 2022] POST /identity/v3/projects => generated 379 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 10:38:50.963200 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.auth.core [None req-d3427fe3-d9e7-476d-9d06-6a7c0a731b03 None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:38:50.964343 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb040f07-fa38-4592-8510-d40ff0fdebbe admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:38:50.965724 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-d3427fe3-d9e7-476d-9d06-6a7c0a731b03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:38:51.049216 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb040f07-fa38-4592-8510-d40ff0fdebbe admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:38:51.054754 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efdfb60d-1aff-4eca-affc-bdd1cdeeba9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:38:51.055603 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdfb60d-1aff-4eca-affc-bdd1cdeeba9e admin admin] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:38:51.055942 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdfb60d-1aff-4eca-affc-bdd1cdeeba9e admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:38:51.056232 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdfb60d-1aff-4eca-affc-bdd1cdeeba9e admin admin] PATH_INFO: `/v3/users` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:38:51.056953 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efdfb60d-1aff-4eca-affc-bdd1cdeeba9e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:38:51.058904 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efdfb60d-1aff-4eca-affc-bdd1cdeeba9e admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:38:51.059501 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1369/2728] 10.222.0.49 () {58 vars in 957 bytes} [Wed Jun 15 10:38:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:38:51.072261 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3427fe3-d9e7-476d-9d06-6a7c0a731b03 None None] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:38:51.074233 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-d3427fe3-d9e7-476d-9d06-6a7c0a731b03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:38:51.081083 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:38:51.090397 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1360/2729] 10.222.0.49 () {60 vars in 1143 bytes} [Wed Jun 15 10:38:50 2022] POST /identity/v3/users => generated 290 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 10:38:51.095395 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efdfb60d-1aff-4eca-affc-bdd1cdeeba9e admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:38:51.112596 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efdfb60d-1aff-4eca-affc-bdd1cdeeba9e admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:38:51.117622 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044841be-3186-44cf-9c8d-a0b6778c52b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:38:51.118392 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044841be-3186-44cf-9c8d-a0b6778c52b5 admin admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:38:51.118749 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044841be-3186-44cf-9c8d-a0b6778c52b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:38:51.119117 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044841be-3186-44cf-9c8d-a0b6778c52b5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:38:51.119928 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-044841be-3186-44cf-9c8d-a0b6778c52b5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:38:51.121666 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-044841be-3186-44cf-9c8d-a0b6778c52b5 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:38:51.142341 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1361/2730] 10.222.0.49 () {58 vars in 1121 bytes} [Wed Jun 15 10:38:51 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 10:38:51.143386 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044841be-3186-44cf-9c8d-a0b6778c52b5 admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:38:51.170803 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044841be-3186-44cf-9c8d-a0b6778c52b5 admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:38:51.182124 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4beb87d0-e5e9-4e07-b2dd-008050b28f8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:38:51.183062 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3427fe3-d9e7-476d-9d06-6a7c0a731b03 None None] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:38:51.183505 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4beb87d0-e5e9-4e07-b2dd-008050b28f8e admin admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:38:51.184379 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4beb87d0-e5e9-4e07-b2dd-008050b28f8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:38:51.184674 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4beb87d0-e5e9-4e07-b2dd-008050b28f8e admin admin] PATH_INFO: `/v3/roles` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:38:51.186173 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4beb87d0-e5e9-4e07-b2dd-008050b28f8e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:38:51.188368 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4beb87d0-e5e9-4e07-b2dd-008050b28f8e admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:38:51.194363 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1b0c3a-e51c-4aac-8de0-bd3f3e6bc533 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0cb44a17d205431793e9a8791e477a32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '559b3e713f5844b6ae37151d56dd419f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:38:51.195128 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1b0c3a-e51c-4aac-8de0-bd3f3e6bc533 service neutron] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:38:51.195397 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1b0c3a-e51c-4aac-8de0-bd3f3e6bc533 service neutron] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:38:51.195675 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1b0c3a-e51c-4aac-8de0-bd3f3e6bc533 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:38:51.214170 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1362/2731] 10.222.0.49 () {58 vars in 1121 bytes} [Wed Jun 15 10:38:51 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 10:38:51.218086 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4beb87d0-e5e9-4e07-b2dd-008050b28f8e admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:38:51.219666 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e1b0c3a-e51c-4aac-8de0-bd3f3e6bc533 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:38:51.221435 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e1b0c3a-e51c-4aac-8de0-bd3f3e6bc533 service neutron] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:38:51.221592 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-4beb87d0-e5e9-4e07-b2dd-008050b28f8e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:38:51.237879 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4beb87d0-e5e9-4e07-b2dd-008050b28f8e admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:38:51.244506 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4dbaea3-2a22-4d95-909c-8ef616b366ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:38:51.245681 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4dbaea3-2a22-4d95-909c-8ef616b366ca admin admin] REQUEST_METHOD: `DELETE` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:38:51.246122 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4dbaea3-2a22-4d95-909c-8ef616b366ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:38:51.246552 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4dbaea3-2a22-4d95-909c-8ef616b366ca admin admin] PATH_INFO: `/v3/users/8714c96315fe434584ccdaf2893b8ad4` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:38:51.248963 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4dbaea3-2a22-4d95-909c-8ef616b366ca admin admin] RBAC: Authorizing `identity:delete_user(user_id=8714c96315fe434584ccdaf2893b8ad4)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:38:51.251343 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4dbaea3-2a22-4d95-909c-8ef616b366ca admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:38:51.255174 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1370/2732] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Jun 15 10:38:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 10:38:51.256204 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1b0c3a-e51c-4aac-8de0-bd3f3e6bc533 service neutron] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:38:51.269325 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1b0c3a-e51c-4aac-8de0-bd3f3e6bc533 service neutron] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:38:51.286881 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8e8a09-29e1-43b8-8329-ec415876013d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:38:51.288215 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8e8a09-29e1-43b8-8329-ec415876013d admin admin] REQUEST_METHOD: `PUT` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:38:51.289276 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8e8a09-29e1-43b8-8329-ec415876013d admin admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:38:51.290374 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8e8a09-29e1-43b8-8329-ec415876013d admin admin] PATH_INFO: `/v3/projects/36a6fb075f3b434d9ff5c7b0c68b85f1/users/5d30c6b3a0234be39a7f24929d0ba1c2/roles/6876e57cc51c448097ab109d35b2a3da` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:38:51.340400 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d8e8a09-29e1-43b8-8329-ec415876013d admin admin] RBAC: Authorizing `identity:create_grant(project_id=36a6fb075f3b434d9ff5c7b0c68b85f1, user_id=5d30c6b3a0234be39a7f24929d0ba1c2, role_id=6876e57cc51c448097ab109d35b2a3da)` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:38:51.348020 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d8e8a09-29e1-43b8-8329-ec415876013d admin admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:38:51.365413 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-e4dbaea3-2a22-4d95-909c-8ef616b366ca admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8714c96315fe434584ccdaf2893b8ad4'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:38:51.384501 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1371/2733] 10.222.0.49 () {60 vars in 1482 bytes} [Wed Jun 15 10:38:51 2022] PUT /identity/v3/projects/36a6fb075f3b434d9ff5c7b0c68b85f1/users/5d30c6b3a0234be39a7f24929d0ba1c2/roles/6876e57cc51c448097ab109d35b2a3da => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:38:51.386820 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-e4dbaea3-2a22-4d95-909c-8ef616b366ca admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8714c96315fe434584ccdaf2893b8ad4'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:38:51.388155 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-e4dbaea3-2a22-4d95-909c-8ef616b366ca admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8714c96315fe434584ccdaf2893b8ad4'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:38:51.388999 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-e4dbaea3-2a22-4d95-909c-8ef616b366ca admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8714c96315fe434584ccdaf2893b8ad4'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:38:51.396274 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a778cf76-5c0e-4ab1-b884-58ab028f9de5 None None] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:38:51.396825 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a778cf76-5c0e-4ab1-b884-58ab028f9de5 None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:38:51.398381 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a778cf76-5c0e-4ab1-b884-58ab028f9de5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:38:51.405958 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-e4dbaea3-2a22-4d95-909c-8ef616b366ca admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8714c96315fe434584ccdaf2893b8ad4'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:38:51.448869 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1363/2734] 10.222.0.49 () {58 vars in 1223 bytes} [Wed Jun 15 10:38:51 2022] DELETE /identity/v3/users/8714c96315fe434584ccdaf2893b8ad4 => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:38:51.454109 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f7058e-a207-4c28-aea2-438bc4de7fe5 None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:38:51.454518 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f7058e-a207-4c28-aea2-438bc4de7fe5 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:38:51.454897 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f7058e-a207-4c28-aea2-438bc4de7fe5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:38:51.464914 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.auth.core [None req-a778cf76-5c0e-4ab1-b884-58ab028f9de5 None None] MFA Rules not processed for user `5d30c6b3a0234be39a7f24929d0ba1c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:38:51.469594 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-a778cf76-5c0e-4ab1-b884-58ab028f9de5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:38:51.479334 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-73f7058e-a207-4c28-aea2-438bc4de7fe5 None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:38:51.480763 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-73f7058e-a207-4c28-aea2-438bc4de7fe5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:38:51.507078 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=85,keystone:INSERT=5,keystone:DELETE=10 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:38:51.571243 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1364/2735] 10.222.0.49 () {58 vars in 957 bytes} [Wed Jun 15 10:38:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:38:51.581422 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f7058e-a207-4c28-aea2-438bc4de7fe5 None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:38:51.582668 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-73f7058e-a207-4c28-aea2-438bc4de7fe5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:38:51.594496 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1372/2736] 10.222.0.49 () {58 vars in 957 bytes} [Wed Jun 15 10:38:51 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:38:51.599282 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a778cf76-5c0e-4ab1-b884-58ab028f9de5 None None] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:38:51.601347 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-a778cf76-5c0e-4ab1-b884-58ab028f9de5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:38:51.622179 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a778cf76-5c0e-4ab1-b884-58ab028f9de5 None None] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:38:51.628201 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f651a65f-5169-4900-9fea-11ac527eec07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:38:51.629318 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f651a65f-5169-4900-9fea-11ac527eec07 admin admin] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:38:51.629703 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f651a65f-5169-4900-9fea-11ac527eec07 admin admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:38:51.630105 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f651a65f-5169-4900-9fea-11ac527eec07 admin admin] PATH_INFO: `/v3/projects` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:38:51.631065 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f651a65f-5169-4900-9fea-11ac527eec07 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:38:51.633148 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f651a65f-5169-4900-9fea-11ac527eec07 admin admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:38:51.674973 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1373/2737] 10.222.0.49 () {60 vars in 1152 bytes} [Wed Jun 15 10:38:51 2022] POST /identity/v3/projects => generated 377 bytes in 76 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 10:38:51.679513 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f651a65f-5169-4900-9fea-11ac527eec07 admin admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:38:51.680586 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f7058e-a207-4c28-aea2-438bc4de7fe5 None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:38:51.784531 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799f6382-1926-473f-85d1-8fb22168489a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0cb44a17d205431793e9a8791e477a32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '559b3e713f5844b6ae37151d56dd419f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:38:51.785344 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:38:51.786846 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799f6382-1926-473f-85d1-8fb22168489a service neutron] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:38:51.787457 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799f6382-1926-473f-85d1-8fb22168489a service neutron] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:38:51.787790 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799f6382-1926-473f-85d1-8fb22168489a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:38:51.820231 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f651a65f-5169-4900-9fea-11ac527eec07 admin admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:38:51.826620 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a25bc5b-1a1c-44d0-b51a-ea4a7847c6b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:38:51.827367 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a25bc5b-1a1c-44d0-b51a-ea4a7847c6b7 admin admin] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:38:51.827746 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a25bc5b-1a1c-44d0-b51a-ea4a7847c6b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:38:51.828113 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a25bc5b-1a1c-44d0-b51a-ea4a7847c6b7 admin admin] PATH_INFO: `/v3/users` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:38:51.828962 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a25bc5b-1a1c-44d0-b51a-ea4a7847c6b7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:38:51.831279 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a25bc5b-1a1c-44d0-b51a-ea4a7847c6b7 admin admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:38:51.865811 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1374/2738] 10.222.0.49 () {60 vars in 1143 bytes} [Wed Jun 15 10:38:51 2022] POST /identity/v3/users => generated 289 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 10:38:51.866933 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a25bc5b-1a1c-44d0-b51a-ea4a7847c6b7 admin admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:38:51.885523 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a25bc5b-1a1c-44d0-b51a-ea4a7847c6b7 admin admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:38:51.893391 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32964910-b9ea-4c22-8dcc-1491b5621150 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:38:51.894368 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32964910-b9ea-4c22-8dcc-1491b5621150 admin admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:38:51.894808 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32964910-b9ea-4c22-8dcc-1491b5621150 admin admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:38:51.895492 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32964910-b9ea-4c22-8dcc-1491b5621150 admin admin] PATH_INFO: `/v3/roles` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:38:51.897789 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32964910-b9ea-4c22-8dcc-1491b5621150 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:38:51.902823 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32964910-b9ea-4c22-8dcc-1491b5621150 admin admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:38:51.916551 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799f6382-1926-473f-85d1-8fb22168489a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:38:51.918213 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799f6382-1926-473f-85d1-8fb22168489a service neutron] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:38:51.919244 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1375/2739] 10.222.0.49 () {58 vars in 1121 bytes} [Wed Jun 15 10:38:51 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 10:38:51.920298 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32964910-b9ea-4c22-8dcc-1491b5621150 admin admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:38:51.923258 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-32964910-b9ea-4c22-8dcc-1491b5621150 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:38:51.932645 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1365/2740] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Jun 15 10:38:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:38:51.933659 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799f6382-1926-473f-85d1-8fb22168489a service neutron] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:38:51.943644 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32964910-b9ea-4c22-8dcc-1491b5621150 admin admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:38:51.946488 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799f6382-1926-473f-85d1-8fb22168489a service neutron] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:38:51.952985 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cadd177-5d3a-4dc9-9037-d185c929d874 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:38:51.953982 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a2dfa1-ef3d-4156-9e0f-2fcc739837e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:38:51.954683 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a2dfa1-ef3d-4156-9e0f-2fcc739837e6 admin admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:38:51.954972 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a2dfa1-ef3d-4156-9e0f-2fcc739837e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:38:51.955348 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cadd177-5d3a-4dc9-9037-d185c929d874 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:38:51.955468 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a2dfa1-ef3d-4156-9e0f-2fcc739837e6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:38:51.956041 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cadd177-5d3a-4dc9-9037-d185c929d874 admin admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:38:51.956212 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a2dfa1-ef3d-4156-9e0f-2fcc739837e6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:38:51.957030 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cadd177-5d3a-4dc9-9037-d185c929d874 admin admin] PATH_INFO: `/v3/projects/e6b481bd220347a7a8843e470264615d` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:38:51.958430 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a2dfa1-ef3d-4156-9e0f-2fcc739837e6 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:38:51.959547 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cadd177-5d3a-4dc9-9037-d185c929d874 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e6b481bd220347a7a8843e470264615d)` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:38:51.962933 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cadd177-5d3a-4dc9-9037-d185c929d874 admin admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:38:51.972016 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1366/2741] 10.222.0.49 () {58 vars in 1121 bytes} [Wed Jun 15 10:38:51 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 10:38:51.975930 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a2dfa1-ef3d-4156-9e0f-2fcc739837e6 admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:38:51.988708 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a2dfa1-ef3d-4156-9e0f-2fcc739837e6 admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:38:51.994720 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0974cf-cab8-4c57-a030-398cd93f4197 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:38:51.995809 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0974cf-cab8-4c57-a030-398cd93f4197 admin admin] REQUEST_METHOD: `PUT` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:38:51.996376 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0974cf-cab8-4c57-a030-398cd93f4197 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:38:51.996832 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0974cf-cab8-4c57-a030-398cd93f4197 admin admin] PATH_INFO: `/v3/projects/776e1310de8b40a9819d42b46476c9f2/users/11a093c78dda47f8b4b707761121ef65/roles/6876e57cc51c448097ab109d35b2a3da` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:38:52.018171 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0974cf-cab8-4c57-a030-398cd93f4197 admin admin] RBAC: Authorizing `identity:create_grant(project_id=776e1310de8b40a9819d42b46476c9f2, user_id=11a093c78dda47f8b4b707761121ef65, role_id=6876e57cc51c448097ab109d35b2a3da)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:38:52.022407 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0974cf-cab8-4c57-a030-398cd93f4197 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:38:52.045101 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:38:52.057178 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1367/2742] 10.222.0.49 () {60 vars in 1482 bytes} [Wed Jun 15 10:38:51 2022] PUT /identity/v3/projects/776e1310de8b40a9819d42b46476c9f2/users/11a093c78dda47f8b4b707761121ef65/roles/6876e57cc51c448097ab109d35b2a3da => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 10:38:52.067935 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced20436-abf0-484b-886a-8219f4076fa7 None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:38:52.067935 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced20436-abf0-484b-886a-8219f4076fa7 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:38:52.068741 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced20436-abf0-484b-886a-8219f4076fa7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:38:52.087049 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-6cadd177-5d3a-4dc9-9037-d185c929d874 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e6b481bd220347a7a8843e470264615d', 'request_id': 'req-6cadd177-5d3a-4dc9-9037-d185c929d874'} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:38:52.100499 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-6cadd177-5d3a-4dc9-9037-d185c929d874 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e6b481bd220347a7a8843e470264615d', 'request_id': 'req-6cadd177-5d3a-4dc9-9037-d185c929d874'} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:38:52.110261 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-6cadd177-5d3a-4dc9-9037-d185c929d874 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e6b481bd220347a7a8843e470264615d', 'request_id': 'req-6cadd177-5d3a-4dc9-9037-d185c929d874'} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:38:52.116455 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-ced20436-abf0-484b-886a-8219f4076fa7 None None] MFA Rules not processed for user `11a093c78dda47f8b4b707761121ef65`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:38:52.118675 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-ced20436-abf0-484b-886a-8219f4076fa7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:38:52.138362 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-6cadd177-5d3a-4dc9-9037-d185c929d874 admin admin] The token cache is being invalidate because project e6b481bd220347a7a8843e470264615d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80119) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 10:38:52.138664 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-6cadd177-5d3a-4dc9-9037-d185c929d874 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:38:52.142643 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-6cadd177-5d3a-4dc9-9037-d185c929d874 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:38:52.144401 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1376/2743] 10.222.0.49 () {58 vars in 1232 bytes} [Wed Jun 15 10:38:51 2022] DELETE /identity/v3/projects/e6b481bd220347a7a8843e470264615d => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:38:52.152887 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2becc8-56fc-43ac-aac0-ee79f77520b2 None None] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:38:52.153178 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2becc8-56fc-43ac-aac0-ee79f77520b2 None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:38:52.153461 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2becc8-56fc-43ac-aac0-ee79f77520b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:38:52.182237 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.auth.core [None req-9c2becc8-56fc-43ac-aac0-ee79f77520b2 None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:38:52.185093 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-9c2becc8-56fc-43ac-aac0-ee79f77520b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:38:52.276863 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1368/2744] 10.222.0.49 () {58 vars in 957 bytes} [Wed Jun 15 10:38:52 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 10:38:52.281748 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced20436-abf0-484b-886a-8219f4076fa7 None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:38:52.306962 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-ced20436-abf0-484b-886a-8219f4076fa7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:38:52.361053 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1377/2745] 10.222.0.49 () {58 vars in 957 bytes} [Wed Jun 15 10:38:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:38:52.369520 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd52f16f-559f-4e8e-ac14-9275369ce204 None None] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:38:52.377554 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd52f16f-559f-4e8e-ac14-9275369ce204 None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:38:52.378999 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd52f16f-559f-4e8e-ac14-9275369ce204 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:38:52.428708 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced20436-abf0-484b-886a-8219f4076fa7 None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:38:52.434391 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.auth.core [None req-fd52f16f-559f-4e8e-ac14-9275369ce204 None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:38:52.435019 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf551ff-638c-4550-a65d-42f54a38ac07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:38:52.436281 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-fd52f16f-559f-4e8e-ac14-9275369ce204 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:38:52.436451 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf551ff-638c-4550-a65d-42f54a38ac07 admin admin] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:38:52.436701 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf551ff-638c-4550-a65d-42f54a38ac07 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:38:52.436982 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf551ff-638c-4550-a65d-42f54a38ac07 admin admin] PATH_INFO: `/v3/projects` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:38:52.437728 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf551ff-638c-4550-a65d-42f54a38ac07 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:38:52.440714 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf551ff-638c-4550-a65d-42f54a38ac07 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:38:52.452508 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1378/2746] 10.222.0.49 () {58 vars in 957 bytes} [Wed Jun 15 10:38:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:38:52.458850 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd52f16f-559f-4e8e-ac14-9275369ce204 None None] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:38:52.493525 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd52f16f-559f-4e8e-ac14-9275369ce204 None None] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:38:52.501995 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1369/2747] 10.222.0.49 () {60 vars in 1152 bytes} [Wed Jun 15 10:38:52 2022] POST /identity/v3/projects => generated 379 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 10:38:52.502891 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34f2472-b8bd-426c-a669-b19ff54dfcaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:38:52.503023 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf551ff-638c-4550-a65d-42f54a38ac07 admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:38:52.504033 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34f2472-b8bd-426c-a669-b19ff54dfcaf admin admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:38:52.504329 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34f2472-b8bd-426c-a669-b19ff54dfcaf admin admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:38:52.504611 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34f2472-b8bd-426c-a669-b19ff54dfcaf admin admin] PATH_INFO: `/v3/domains` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:38:52.504719 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-aaf551ff-638c-4550-a65d-42f54a38ac07 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:38:52.505351 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34f2472-b8bd-426c-a669-b19ff54dfcaf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:38:52.505659 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34f2472-b8bd-426c-a669-b19ff54dfcaf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:38:52.509293 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34f2472-b8bd-426c-a669-b19ff54dfcaf admin admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:38:52.542145 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1379/2748] 10.222.0.49 () {58 vars in 1152 bytes} [Wed Jun 15 10:38:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 10:38:52.544640 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32544a0-2da0-40f6-b9e8-eb482cd9d482 None None] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:38:52.544928 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32544a0-2da0-40f6-b9e8-eb482cd9d482 None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:38:52.545213 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32544a0-2da0-40f6-b9e8-eb482cd9d482 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:38:52.573853 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.auth.core [None req-f32544a0-2da0-40f6-b9e8-eb482cd9d482 None None] MFA Rules not processed for user `43929157365d4fad979f66e227572393`. Rule list: `[]` (Enabled: `True`). {{(pid=80119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:38:52.575010 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-f32544a0-2da0-40f6-b9e8-eb482cd9d482 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:38:52.597542 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf551ff-638c-4550-a65d-42f54a38ac07 admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:38:52.598485 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:38:52.602713 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76249510-4e96-4503-9ce0-093cfdc2f082 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:38:52.605345 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76249510-4e96-4503-9ce0-093cfdc2f082 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:38:52.605630 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76249510-4e96-4503-9ce0-093cfdc2f082 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:38:52.605911 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76249510-4e96-4503-9ce0-093cfdc2f082 admin admin] PATH_INFO: `/v3/projects/c0896c3d67384b1493e5e6287b8ae9e7` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:38:52.607569 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76249510-4e96-4503-9ce0-093cfdc2f082 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c0896c3d67384b1493e5e6287b8ae9e7)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:38:52.610079 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76249510-4e96-4503-9ce0-093cfdc2f082 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:38:52.636680 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:38:52.670100 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1380/2749] 10.222.0.49 () {58 vars in 957 bytes} [Wed Jun 15 10:38:52 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:38:52.670920 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32544a0-2da0-40f6-b9e8-eb482cd9d482 None None] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:38:52.683809 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32544a0-2da0-40f6-b9e8-eb482cd9d482 None None] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:38:52.690152 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-76249510-4e96-4503-9ce0-093cfdc2f082 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c0896c3d67384b1493e5e6287b8ae9e7', 'request_id': 'req-76249510-4e96-4503-9ce0-093cfdc2f082'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:38:52.700901 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-76249510-4e96-4503-9ce0-093cfdc2f082 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c0896c3d67384b1493e5e6287b8ae9e7', 'request_id': 'req-76249510-4e96-4503-9ce0-093cfdc2f082'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:38:52.708211 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-76249510-4e96-4503-9ce0-093cfdc2f082 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c0896c3d67384b1493e5e6287b8ae9e7', 'request_id': 'req-76249510-4e96-4503-9ce0-093cfdc2f082'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:38:52.723518 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-76249510-4e96-4503-9ce0-093cfdc2f082 admin admin] The token cache is being invalidate because project c0896c3d67384b1493e5e6287b8ae9e7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80120) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 10:38:52.723807 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-76249510-4e96-4503-9ce0-093cfdc2f082 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:38:52.724649 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-76249510-4e96-4503-9ce0-093cfdc2f082 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:38:52.726233 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1370/2750] 10.222.0.49 () {58 vars in 1232 bytes} [Wed Jun 15 10:38:52 2022] DELETE /identity/v3/projects/c0896c3d67384b1493e5e6287b8ae9e7 => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:38:52.727194 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76249510-4e96-4503-9ce0-093cfdc2f082 admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:38:52.728546 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-76249510-4e96-4503-9ce0-093cfdc2f082 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:38:52.756261 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b941f0b0-4026-46f3-a555-b5b65a377d84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:38:52.757049 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b941f0b0-4026-46f3-a555-b5b65a377d84 admin admin] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:38:52.757461 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b941f0b0-4026-46f3-a555-b5b65a377d84 admin admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:38:52.757817 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b941f0b0-4026-46f3-a555-b5b65a377d84 admin admin] PATH_INFO: `/v3/users` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:38:52.758571 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b941f0b0-4026-46f3-a555-b5b65a377d84 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:38:52.760884 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b941f0b0-4026-46f3-a555-b5b65a377d84 admin admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:38:52.790649 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1381/2751] 10.222.0.49 () {60 vars in 1143 bytes} [Wed Jun 15 10:38:52 2022] POST /identity/v3/users => generated 296 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 10:38:52.791754 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b941f0b0-4026-46f3-a555-b5b65a377d84 admin admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:38:52.793806 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-b941f0b0-4026-46f3-a555-b5b65a377d84 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:38:52.811920 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76249510-4e96-4503-9ce0-093cfdc2f082 admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:38:52.818808 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44367d36-c951-42f6-a0c2-a4761d3c2df6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:38:52.819601 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44367d36-c951-42f6-a0c2-a4761d3c2df6 admin admin] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:38:52.820173 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44367d36-c951-42f6-a0c2-a4761d3c2df6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:38:52.820459 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44367d36-c951-42f6-a0c2-a4761d3c2df6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:38:52.821162 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44367d36-c951-42f6-a0c2-a4761d3c2df6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:38:52.823112 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44367d36-c951-42f6-a0c2-a4761d3c2df6 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:38:52.858243 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1371/2752] 10.222.0.49 () {60 vars in 1152 bytes} [Wed Jun 15 10:38:52 2022] POST /identity/v3/projects => generated 373 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 10:38:52.858243 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44367d36-c951-42f6-a0c2-a4761d3c2df6 admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:38:52.858243 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-44367d36-c951-42f6-a0c2-a4761d3c2df6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:38:52.882089 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b941f0b0-4026-46f3-a555-b5b65a377d84 admin admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:38:52.951205 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:38:52.963971 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44367d36-c951-42f6-a0c2-a4761d3c2df6 admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:38:52.969643 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0c40f8-7cda-43ce-a90d-ff80b9335989 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:38:52.970543 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0c40f8-7cda-43ce-a90d-ff80b9335989 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:38:52.970830 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0c40f8-7cda-43ce-a90d-ff80b9335989 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:38:52.971142 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0c40f8-7cda-43ce-a90d-ff80b9335989 admin admin] PATH_INFO: `/v3/users/9771429b486c4bce9d48dd246273d031` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:38:52.972384 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0c40f8-7cda-43ce-a90d-ff80b9335989 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9771429b486c4bce9d48dd246273d031)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:38:52.974329 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0c40f8-7cda-43ce-a90d-ff80b9335989 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:38:52.979858 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7808d0-5926-423f-aa36-083eb1a5a324 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ac54528370d4747b6e8777fba9d0c82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '559b3e713f5844b6ae37151d56dd419f', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:38:52.981413 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7808d0-5926-423f-aa36-083eb1a5a324 service nova] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:38:52.981873 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7808d0-5926-423f-aa36-083eb1a5a324 service nova] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:38:52.982213 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7808d0-5926-423f-aa36-083eb1a5a324 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:38:52.984343 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-7c7808d0-5926-423f-aa36-083eb1a5a324 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:38:53.014196 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:38:53.037751 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-6e0c40f8-7cda-43ce-a90d-ff80b9335989 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9771429b486c4bce9d48dd246273d031'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:38:53.045588 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-6e0c40f8-7cda-43ce-a90d-ff80b9335989 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9771429b486c4bce9d48dd246273d031'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:38:53.046398 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-6e0c40f8-7cda-43ce-a90d-ff80b9335989 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9771429b486c4bce9d48dd246273d031'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:38:53.047282 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-6e0c40f8-7cda-43ce-a90d-ff80b9335989 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9771429b486c4bce9d48dd246273d031'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:38:53.056079 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-6e0c40f8-7cda-43ce-a90d-ff80b9335989 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9771429b486c4bce9d48dd246273d031'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:38:53.066339 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7808d0-5926-423f-aa36-083eb1a5a324 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:38:53.068146 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7808d0-5926-423f-aa36-083eb1a5a324 service nova] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:38:53.080359 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1382/2753] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Jun 15 10:38:52 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 289 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:38:53.081208 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7808d0-5926-423f-aa36-083eb1a5a324 service nova] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:38:53.082902 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-7c7808d0-5926-423f-aa36-083eb1a5a324 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:38:53.086843 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1372/2754] 10.222.0.49 () {58 vars in 1223 bytes} [Wed Jun 15 10:38:52 2022] DELETE /identity/v3/users/9771429b486c4bce9d48dd246273d031 => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:38:53.086843 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0c40f8-7cda-43ce-a90d-ff80b9335989 admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:38:53.087143 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-6e0c40f8-7cda-43ce-a90d-ff80b9335989 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:38:53.123239 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7808d0-5926-423f-aa36-083eb1a5a324 service nova] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:38:53.169759 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0c40f8-7cda-43ce-a90d-ff80b9335989 admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:38:53.176962 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ea7320-b29d-4fd1-b2af-bf9e8f3d6f79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:38:53.177644 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ea7320-b29d-4fd1-b2af-bf9e8f3d6f79 admin admin] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:38:53.177988 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ea7320-b29d-4fd1-b2af-bf9e8f3d6f79 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:38:53.178341 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ea7320-b29d-4fd1-b2af-bf9e8f3d6f79 admin admin] PATH_INFO: `/v3/users` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:38:53.180553 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ea7320-b29d-4fd1-b2af-bf9e8f3d6f79 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:38:53.183053 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ea7320-b29d-4fd1-b2af-bf9e8f3d6f79 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:38:53.211826 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1373/2755] 10.222.0.49 () {60 vars in 1143 bytes} [Wed Jun 15 10:38:53 2022] POST /identity/v3/users => generated 287 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 10:38:53.216588 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ea7320-b29d-4fd1-b2af-bf9e8f3d6f79 admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:38:53.233608 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ea7320-b29d-4fd1-b2af-bf9e8f3d6f79 admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:38:53.245917 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2edb6fe9-aaf6-485b-bcbb-59ae1c4cdf1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:38:53.248446 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edb6fe9-aaf6-485b-bcbb-59ae1c4cdf1a admin admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:38:53.251115 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edb6fe9-aaf6-485b-bcbb-59ae1c4cdf1a admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:38:53.251516 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edb6fe9-aaf6-485b-bcbb-59ae1c4cdf1a admin admin] PATH_INFO: `/v3/roles` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:38:53.253923 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2edb6fe9-aaf6-485b-bcbb-59ae1c4cdf1a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:38:53.255738 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2edb6fe9-aaf6-485b-bcbb-59ae1c4cdf1a admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:38:53.267542 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ea15ec-7e83-44de-931d-006dcf59fdd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:38:53.268362 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ea15ec-7e83-44de-931d-006dcf59fdd1 admin admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:38:53.268785 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ea15ec-7e83-44de-931d-006dcf59fdd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:38:53.269213 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ea15ec-7e83-44de-931d-006dcf59fdd1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:38:53.270146 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1374/2756] 10.222.0.49 () {58 vars in 1121 bytes} [Wed Jun 15 10:38:53 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 10:38:53.273669 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ea15ec-7e83-44de-931d-006dcf59fdd1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:38:53.276667 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ea15ec-7e83-44de-931d-006dcf59fdd1 admin admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:38:53.277341 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2edb6fe9-aaf6-485b-bcbb-59ae1c4cdf1a admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:38:53.292515 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2edb6fe9-aaf6-485b-bcbb-59ae1c4cdf1a admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:38:53.295661 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1383/2757] 10.222.0.49 () {58 vars in 1121 bytes} [Wed Jun 15 10:38:53 2022] GET /identity/v3/roles => generated 1420 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 10:38:53.300771 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ea15ec-7e83-44de-931d-006dcf59fdd1 admin admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:38:53.303303 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0166746-7993-4b0c-a120-e8bbdb9bc37b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:38:53.303911 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-43ea15ec-7e83-44de-931d-006dcf59fdd1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:38:53.304670 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0166746-7993-4b0c-a120-e8bbdb9bc37b admin admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:38:53.304960 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0166746-7993-4b0c-a120-e8bbdb9bc37b admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:38:53.305237 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0166746-7993-4b0c-a120-e8bbdb9bc37b admin admin] PATH_INFO: `/v3/roles` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:38:53.306038 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0166746-7993-4b0c-a120-e8bbdb9bc37b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:38:53.307800 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0166746-7993-4b0c-a120-e8bbdb9bc37b admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:38:53.341921 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1375/2758] 10.222.0.49 () {58 vars in 1121 bytes} [Wed Jun 15 10:38:53 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 10:38:53.347577 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0166746-7993-4b0c-a120-e8bbdb9bc37b admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:38:53.349083 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ea15ec-7e83-44de-931d-006dcf59fdd1 admin admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:38:53.355901 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7817296c-1620-419d-9727-a6f6421fd919 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:38:53.357858 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7817296c-1620-419d-9727-a6f6421fd919 admin admin] REQUEST_METHOD: `PUT` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:38:53.358140 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7817296c-1620-419d-9727-a6f6421fd919 admin admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:38:53.358429 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7817296c-1620-419d-9727-a6f6421fd919 admin admin] PATH_INFO: `/v3/projects/d96dae0df643424590cd7b65167f3b87/users/c786e2e9feec43d5b57ce496d7074121/roles/8051174e7a7947d5b991ebd793c2e543` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:38:53.362261 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0166746-7993-4b0c-a120-e8bbdb9bc37b admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:38:53.369744 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a4ae61-e506-42ca-bc0e-d8b647db1f5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:38:53.370942 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a4ae61-e506-42ca-bc0e-d8b647db1f5b admin admin] REQUEST_METHOD: `PUT` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:38:53.371436 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a4ae61-e506-42ca-bc0e-d8b647db1f5b admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:38:53.371921 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a4ae61-e506-42ca-bc0e-d8b647db1f5b admin admin] PATH_INFO: `/v3/projects/aab8980da8444a9789dd0c0b4b88aeac/users/8a8ea7b3022e44f48179326c25646c38/roles/6876e57cc51c448097ab109d35b2a3da` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:38:53.382979 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7817296c-1620-419d-9727-a6f6421fd919 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d96dae0df643424590cd7b65167f3b87, user_id=c786e2e9feec43d5b57ce496d7074121, role_id=8051174e7a7947d5b991ebd793c2e543)` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:38:53.388010 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7817296c-1620-419d-9727-a6f6421fd919 admin admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:38:53.398443 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a4ae61-e506-42ca-bc0e-d8b647db1f5b admin admin] RBAC: Authorizing `identity:create_grant(project_id=aab8980da8444a9789dd0c0b4b88aeac, user_id=8a8ea7b3022e44f48179326c25646c38, role_id=6876e57cc51c448097ab109d35b2a3da)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:38:53.406862 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1384/2759] 10.222.0.49 () {60 vars in 1482 bytes} [Wed Jun 15 10:38:53 2022] PUT /identity/v3/projects/d96dae0df643424590cd7b65167f3b87/users/c786e2e9feec43d5b57ce496d7074121/roles/8051174e7a7947d5b991ebd793c2e543 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:38:53.411416 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7817296c-1620-419d-9727-a6f6421fd919 admin admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:38:53.415055 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a4ae61-e506-42ca-bc0e-d8b647db1f5b admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:38:53.434888 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1376/2760] 10.222.0.49 () {60 vars in 1482 bytes} [Wed Jun 15 10:38:53 2022] PUT /identity/v3/projects/aab8980da8444a9789dd0c0b4b88aeac/users/8a8ea7b3022e44f48179326c25646c38/roles/6876e57cc51c448097ab109d35b2a3da => generated 0 bytes in 92 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:38:53.449180 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a013848-aea5-4ad8-8078-231b7f251eab None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:38:53.449771 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a013848-aea5-4ad8-8078-231b7f251eab None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:38:53.450783 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a013848-aea5-4ad8-8078-231b7f251eab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:38:53.485313 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-9a013848-aea5-4ad8-8078-231b7f251eab None None] MFA Rules not processed for user `8a8ea7b3022e44f48179326c25646c38`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:38:53.486603 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-9a013848-aea5-4ad8-8078-231b7f251eab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:38:53.494918 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:38:53.514151 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7817296c-1620-419d-9727-a6f6421fd919 admin admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:38:53.554565 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=90,keystone:DELETE=5,keystone:INSERT=5 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:38:53.593546 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a898b1-d84f-4967-a0b3-5df146610155 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:38:53.594879 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a898b1-d84f-4967-a0b3-5df146610155 admin admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:38:53.595492 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a898b1-d84f-4967-a0b3-5df146610155 admin admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:38:53.595963 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a898b1-d84f-4967-a0b3-5df146610155 admin admin] PATH_INFO: `/v3/roles` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:38:53.596711 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a898b1-d84f-4967-a0b3-5df146610155 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:38:53.598505 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a898b1-d84f-4967-a0b3-5df146610155 admin admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:38:53.608572 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1377/2761] 10.222.0.49 () {58 vars in 957 bytes} [Wed Jun 15 10:38:53 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:38:53.608572 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a013848-aea5-4ad8-8078-231b7f251eab None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:38:53.608572 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-9a013848-aea5-4ad8-8078-231b7f251eab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:38:53.611641 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1385/2762] 10.222.0.49 () {58 vars in 1121 bytes} [Wed Jun 15 10:38:53 2022] GET /identity/v3/roles => generated 1420 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 10:38:53.617581 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a898b1-d84f-4967-a0b3-5df146610155 admin admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:38:53.642555 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a898b1-d84f-4967-a0b3-5df146610155 admin admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:38:53.651520 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221865db-a395-43ec-8b82-aae75a1d6fad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:38:53.652483 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221865db-a395-43ec-8b82-aae75a1d6fad admin admin] REQUEST_METHOD: `PUT` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:38:53.652773 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221865db-a395-43ec-8b82-aae75a1d6fad admin admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:38:53.653051 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221865db-a395-43ec-8b82-aae75a1d6fad admin admin] PATH_INFO: `/v3/projects/d96dae0df643424590cd7b65167f3b87/users/c786e2e9feec43d5b57ce496d7074121/roles/8051174e7a7947d5b991ebd793c2e543` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:38:53.655118 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-221865db-a395-43ec-8b82-aae75a1d6fad admin admin] RBAC: Authorizing `identity:create_grant(project_id=d96dae0df643424590cd7b65167f3b87, user_id=c786e2e9feec43d5b57ce496d7074121, role_id=8051174e7a7947d5b991ebd793c2e543)` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:38:53.658471 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-221865db-a395-43ec-8b82-aae75a1d6fad admin admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:38:53.671182 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a013848-aea5-4ad8-8078-231b7f251eab None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:38:53.683122 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1386/2763] 10.222.0.49 () {60 vars in 1482 bytes} [Wed Jun 15 10:38:53 2022] PUT /identity/v3/projects/d96dae0df643424590cd7b65167f3b87/users/c786e2e9feec43d5b57ce496d7074121/roles/8051174e7a7947d5b991ebd793c2e543 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:38:53.683122 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221865db-a395-43ec-8b82-aae75a1d6fad admin admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:38:53.683122 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-221865db-a395-43ec-8b82-aae75a1d6fad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:38:53.781245 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221865db-a395-43ec-8b82-aae75a1d6fad admin admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:38:53.797870 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17e8a84-12b8-4abd-92d7-b71e28a9dcb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:38:53.799082 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17e8a84-12b8-4abd-92d7-b71e28a9dcb4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:38:53.799381 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17e8a84-12b8-4abd-92d7-b71e28a9dcb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:38:53.799663 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17e8a84-12b8-4abd-92d7-b71e28a9dcb4 admin admin] PATH_INFO: `/v3/projects/477acc8404da4fad80d87bbc7e36c497` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:38:53.801610 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c17e8a84-12b8-4abd-92d7-b71e28a9dcb4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=477acc8404da4fad80d87bbc7e36c497)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:38:53.801785 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc4cfef-e2b7-4c47-9248-03660c10181e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20d1327553404b80aecee87c78fcb52b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '559b3e713f5844b6ae37151d56dd419f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:38:53.802789 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc4cfef-e2b7-4c47-9248-03660c10181e service cinder] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:38:53.803270 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc4cfef-e2b7-4c47-9248-03660c10181e service cinder] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:38:53.803632 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c17e8a84-12b8-4abd-92d7-b71e28a9dcb4 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:38:53.803921 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc4cfef-e2b7-4c47-9248-03660c10181e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:38:53.880407 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-c17e8a84-12b8-4abd-92d7-b71e28a9dcb4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '477acc8404da4fad80d87bbc7e36c497', 'request_id': 'req-c17e8a84-12b8-4abd-92d7-b71e28a9dcb4'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:38:53.891649 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-c17e8a84-12b8-4abd-92d7-b71e28a9dcb4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '477acc8404da4fad80d87bbc7e36c497', 'request_id': 'req-c17e8a84-12b8-4abd-92d7-b71e28a9dcb4'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:38:53.899923 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-c17e8a84-12b8-4abd-92d7-b71e28a9dcb4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '477acc8404da4fad80d87bbc7e36c497', 'request_id': 'req-c17e8a84-12b8-4abd-92d7-b71e28a9dcb4'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:38:53.907354 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc4cfef-e2b7-4c47-9248-03660c10181e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:38:53.909076 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc4cfef-e2b7-4c47-9248-03660c10181e service cinder] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:38:53.919948 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1387/2764] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Jun 15 10:38:53 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 245 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:38:53.920751 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-c17e8a84-12b8-4abd-92d7-b71e28a9dcb4 admin admin] The token cache is being invalidate because project 477acc8404da4fad80d87bbc7e36c497 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80120) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 10:38:53.921175 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-c17e8a84-12b8-4abd-92d7-b71e28a9dcb4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:38:53.921916 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-c17e8a84-12b8-4abd-92d7-b71e28a9dcb4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:38:53.924134 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555afdda-3ff3-46d5-9edc-5fcbb0abc9c4 None None] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:38:53.924396 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1378/2765] 10.222.0.49 () {58 vars in 1232 bytes} [Wed Jun 15 10:38:53 2022] DELETE /identity/v3/projects/477acc8404da4fad80d87bbc7e36c497 => generated 0 bytes in 318 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:38:53.927679 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555afdda-3ff3-46d5-9edc-5fcbb0abc9c4 None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:38:53.928362 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555afdda-3ff3-46d5-9edc-5fcbb0abc9c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:38:53.929869 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e6cb2c-390a-4967-844a-263973df4aba None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:38:53.930231 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e6cb2c-390a-4967-844a-263973df4aba None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:38:53.930658 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e6cb2c-390a-4967-844a-263973df4aba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:38:53.955236 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-50e6cb2c-390a-4967-844a-263973df4aba None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:38:53.956896 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-50e6cb2c-390a-4967-844a-263973df4aba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:38:53.964352 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.auth.core [None req-555afdda-3ff3-46d5-9edc-5fcbb0abc9c4 None None] MFA Rules not processed for user `c786e2e9feec43d5b57ce496d7074121`. Rule list: `[]` (Enabled: `True`). {{(pid=80119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:38:53.966586 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-555afdda-3ff3-46d5-9edc-5fcbb0abc9c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:38:53.998486 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:38:54.041525 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1379/2766] 10.222.0.49 () {58 vars in 957 bytes} [Wed Jun 15 10:38:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:38:54.050834 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b1198a-5b96-471c-8d36-940ca183a8cd None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:38:54.051493 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b1198a-5b96-471c-8d36-940ca183a8cd None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:38:54.051875 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b1198a-5b96-471c-8d36-940ca183a8cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:38:54.077479 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1388/2767] 10.222.0.49 () {58 vars in 957 bytes} [Wed Jun 15 10:38:53 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:38:54.078823 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-84b1198a-5b96-471c-8d36-940ca183a8cd None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:38:54.080114 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-84b1198a-5b96-471c-8d36-940ca183a8cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:38:54.089235 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555afdda-3ff3-46d5-9edc-5fcbb0abc9c4 None None] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:38:54.090778 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-555afdda-3ff3-46d5-9edc-5fcbb0abc9c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:38:54.091608 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1380/2768] 10.222.0.49 () {58 vars in 957 bytes} [Wed Jun 15 10:38:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:38:54.096381 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b1198a-5b96-471c-8d36-940ca183a8cd None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:38:54.104447 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:38:54.109437 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b1198a-5b96-471c-8d36-940ca183a8cd None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:38:54.128983 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978935d5-d6b1-4f5f-b8a0-22ff60f3b617 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:38:54.129416 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978935d5-d6b1-4f5f-b8a0-22ff60f3b617 admin admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:38:54.129703 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978935d5-d6b1-4f5f-b8a0-22ff60f3b617 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:38:54.129981 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978935d5-d6b1-4f5f-b8a0-22ff60f3b617 admin admin] PATH_INFO: `/v3/domains` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:38:54.130871 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-978935d5-d6b1-4f5f-b8a0-22ff60f3b617 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:38:54.131061 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-978935d5-d6b1-4f5f-b8a0-22ff60f3b617 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:38:54.132736 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-978935d5-d6b1-4f5f-b8a0-22ff60f3b617 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:38:54.148554 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1381/2769] 10.222.0.49 () {58 vars in 1152 bytes} [Wed Jun 15 10:38:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 10:38:54.152720 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978935d5-d6b1-4f5f-b8a0-22ff60f3b617 admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:38:54.166931 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978935d5-d6b1-4f5f-b8a0-22ff60f3b617 admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:38:54.174323 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c379f4f-ebeb-431c-9b2b-531fa1f6c9ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:38:54.175248 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c379f4f-ebeb-431c-9b2b-531fa1f6c9ff admin admin] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:38:54.175559 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c379f4f-ebeb-431c-9b2b-531fa1f6c9ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:38:54.176068 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c379f4f-ebeb-431c-9b2b-531fa1f6c9ff admin admin] PATH_INFO: `/v3/projects` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:38:54.177939 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c379f4f-ebeb-431c-9b2b-531fa1f6c9ff admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:38:54.181175 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c379f4f-ebeb-431c-9b2b-531fa1f6c9ff admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:38:54.195059 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555afdda-3ff3-46d5-9edc-5fcbb0abc9c4 None None] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:38:54.199589 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730628d2-c02e-4ca9-a830-24eb8a2c77cc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20d1327553404b80aecee87c78fcb52b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '559b3e713f5844b6ae37151d56dd419f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:38:54.200460 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730628d2-c02e-4ca9-a830-24eb8a2c77cc service cinder] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:38:54.201003 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730628d2-c02e-4ca9-a830-24eb8a2c77cc service cinder] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:38:54.201561 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730628d2-c02e-4ca9-a830-24eb8a2c77cc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:38:54.215759 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730628d2-c02e-4ca9-a830-24eb8a2c77cc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:38:54.217727 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730628d2-c02e-4ca9-a830-24eb8a2c77cc service cinder] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:38:54.225831 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1382/2770] 10.222.0.49 () {60 vars in 1152 bytes} [Wed Jun 15 10:38:54 2022] POST /identity/v3/projects => generated 373 bytes in 71 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 10:38:54.227253 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c379f4f-ebeb-431c-9b2b-531fa1f6c9ff admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:38:54.233084 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1389/2771] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Jun 15 10:38:54 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 10:38:54.297537 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730628d2-c02e-4ca9-a830-24eb8a2c77cc service cinder] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:38:54.310912 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730628d2-c02e-4ca9-a830-24eb8a2c77cc service cinder] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:38:54.346964 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c379f4f-ebeb-431c-9b2b-531fa1f6c9ff admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:38:54.365946 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604879c4-70ab-488e-8b65-ee2af755c954 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:38:54.366599 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604879c4-70ab-488e-8b65-ee2af755c954 admin admin] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:38:54.366834 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604879c4-70ab-488e-8b65-ee2af755c954 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:38:54.367378 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604879c4-70ab-488e-8b65-ee2af755c954 admin admin] PATH_INFO: `/v3/users` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:38:54.367916 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-604879c4-70ab-488e-8b65-ee2af755c954 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:38:54.369868 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-604879c4-70ab-488e-8b65-ee2af755c954 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:38:54.404984 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ddbf06-2272-4f58-9857-f5e4bc94c0b2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20d1327553404b80aecee87c78fcb52b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '559b3e713f5844b6ae37151d56dd419f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:38:54.405694 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ddbf06-2272-4f58-9857-f5e4bc94c0b2 service cinder] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:38:54.406070 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ddbf06-2272-4f58-9857-f5e4bc94c0b2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:38:54.406508 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ddbf06-2272-4f58-9857-f5e4bc94c0b2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:38:54.408758 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-40ddbf06-2272-4f58-9857-f5e4bc94c0b2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:38:54.409537 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1383/2772] 10.222.0.49 () {60 vars in 1143 bytes} [Wed Jun 15 10:38:54 2022] POST /identity/v3/users => generated 287 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 10:38:54.410621 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604879c4-70ab-488e-8b65-ee2af755c954 admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:38:54.433371 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604879c4-70ab-488e-8b65-ee2af755c954 admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:38:54.455685 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44280d44-39be-46d2-9ecf-ec5d90858921 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:38:54.456561 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44280d44-39be-46d2-9ecf-ec5d90858921 admin admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:38:54.457381 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44280d44-39be-46d2-9ecf-ec5d90858921 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:38:54.457381 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44280d44-39be-46d2-9ecf-ec5d90858921 admin admin] PATH_INFO: `/v3/roles` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:38:54.458216 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44280d44-39be-46d2-9ecf-ec5d90858921 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:38:54.460079 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44280d44-39be-46d2-9ecf-ec5d90858921 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:38:54.473140 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1384/2773] 10.222.0.49 () {58 vars in 1121 bytes} [Wed Jun 15 10:38:54 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 10:38:54.476741 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44280d44-39be-46d2-9ecf-ec5d90858921 admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:38:54.491931 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44280d44-39be-46d2-9ecf-ec5d90858921 admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:38:54.498007 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19d4b83-fed2-478d-81bd-6966df51bffc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:38:54.498931 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19d4b83-fed2-478d-81bd-6966df51bffc admin admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:38:54.499328 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19d4b83-fed2-478d-81bd-6966df51bffc admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:38:54.499680 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19d4b83-fed2-478d-81bd-6966df51bffc admin admin] PATH_INFO: `/v3/roles` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:38:54.501049 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c19d4b83-fed2-478d-81bd-6966df51bffc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:38:54.502904 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c19d4b83-fed2-478d-81bd-6966df51bffc admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:38:54.508260 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ddbf06-2272-4f58-9857-f5e4bc94c0b2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:38:54.509713 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ddbf06-2272-4f58-9857-f5e4bc94c0b2 service cinder] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:38:54.514334 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1385/2774] 10.222.0.49 () {58 vars in 1121 bytes} [Wed Jun 15 10:38:54 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 10:38:54.518341 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19d4b83-fed2-478d-81bd-6966df51bffc admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:38:54.519768 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1390/2775] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Jun 15 10:38:54 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 224 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:38:54.528383 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19d4b83-fed2-478d-81bd-6966df51bffc admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:38:54.535434 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7989c6-6cf2-48e1-aebb-88b78b2c632e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:38:54.536338 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7989c6-6cf2-48e1-aebb-88b78b2c632e admin admin] REQUEST_METHOD: `PUT` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:38:54.536628 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7989c6-6cf2-48e1-aebb-88b78b2c632e admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:38:54.536896 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7989c6-6cf2-48e1-aebb-88b78b2c632e admin admin] PATH_INFO: `/v3/projects/3efa3522561147ac8eff67781403423e/users/07276de63b65403585d0d7c576447f96/roles/6876e57cc51c448097ab109d35b2a3da` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:38:54.555238 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db7989c6-6cf2-48e1-aebb-88b78b2c632e admin admin] RBAC: Authorizing `identity:create_grant(project_id=3efa3522561147ac8eff67781403423e, user_id=07276de63b65403585d0d7c576447f96, role_id=6876e57cc51c448097ab109d35b2a3da)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:38:54.558196 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db7989c6-6cf2-48e1-aebb-88b78b2c632e admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:38:54.578561 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1386/2776] 10.222.0.49 () {60 vars in 1482 bytes} [Wed Jun 15 10:38:54 2022] PUT /identity/v3/projects/3efa3522561147ac8eff67781403423e/users/07276de63b65403585d0d7c576447f96/roles/6876e57cc51c448097ab109d35b2a3da => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:38:54.580874 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce65e555-e6df-433e-a5a1-90236d3b26d0 None None] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:38:54.581169 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce65e555-e6df-433e-a5a1-90236d3b26d0 None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:38:54.581448 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce65e555-e6df-433e-a5a1-90236d3b26d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:38:54.605676 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.auth.core [None req-ce65e555-e6df-433e-a5a1-90236d3b26d0 None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:38:54.606818 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-ce65e555-e6df-433e-a5a1-90236d3b26d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:38:54.622905 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=100 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:38:54.681753 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1391/2777] 10.222.0.49 () {58 vars in 957 bytes} [Wed Jun 15 10:38:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:38:54.693960 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7989c6-6cf2-48e1-aebb-88b78b2c632e admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:38:54.696102 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-db7989c6-6cf2-48e1-aebb-88b78b2c632e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:38:54.778560 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7989c6-6cf2-48e1-aebb-88b78b2c632e admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:38:54.786786 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6edc872d-bf66-49d4-8358-2ae94e78d00a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0cb44a17d205431793e9a8791e477a32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '559b3e713f5844b6ae37151d56dd419f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:38:54.787568 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edc872d-bf66-49d4-8358-2ae94e78d00a service neutron] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:38:54.787857 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edc872d-bf66-49d4-8358-2ae94e78d00a service neutron] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:38:54.788234 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edc872d-bf66-49d4-8358-2ae94e78d00a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:38:54.819932 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6edc872d-bf66-49d4-8358-2ae94e78d00a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:38:54.821599 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6edc872d-bf66-49d4-8358-2ae94e78d00a service neutron] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:38:54.835022 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1387/2778] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Jun 15 10:38:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:38:55.516845 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b7ab28-fbc8-4098-896b-31dfa160fb5f None None] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:38:55.517457 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b7ab28-fbc8-4098-896b-31dfa160fb5f None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:38:55.517579 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b7ab28-fbc8-4098-896b-31dfa160fb5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:38:55.535953 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.auth.core [None req-63b7ab28-fbc8-4098-896b-31dfa160fb5f None None] MFA Rules not processed for user `8a8ea7b3022e44f48179326c25646c38`. Rule list: `[]` (Enabled: `True`). {{(pid=80119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:38:55.537184 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-63b7ab28-fbc8-4098-896b-31dfa160fb5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:38:55.593610 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1392/2779] 10.222.0.49 () {58 vars in 957 bytes} [Wed Jun 15 10:38:55 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:38:55.861454 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49dec52-3b52-418b-b634-532b2e4b78aa None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:38:55.862010 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49dec52-3b52-418b-b634-532b2e4b78aa None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:38:55.862010 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49dec52-3b52-418b-b634-532b2e4b78aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:38:55.899368 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-c49dec52-3b52-418b-b634-532b2e4b78aa None None] MFA Rules not processed for user `07276de63b65403585d0d7c576447f96`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:38:55.901008 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-c49dec52-3b52-418b-b634-532b2e4b78aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:38:55.905594 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:38:55.987557 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1388/2780] 10.222.0.49 () {58 vars in 957 bytes} [Wed Jun 15 10:38:55 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:38:55.999683 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b7ab28-fbc8-4098-896b-31dfa160fb5f None None] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:38:56.002663 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-63b7ab28-fbc8-4098-896b-31dfa160fb5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:38:56.120187 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b7ab28-fbc8-4098-896b-31dfa160fb5f None None] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:38:56.125168 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c05ea4-cec0-4e1f-aa7f-95b12cfddf7e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20d1327553404b80aecee87c78fcb52b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '559b3e713f5844b6ae37151d56dd419f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:38:56.125783 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c05ea4-cec0-4e1f-aa7f-95b12cfddf7e service cinder] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:38:56.126079 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c05ea4-cec0-4e1f-aa7f-95b12cfddf7e service cinder] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:38:56.126363 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c05ea4-cec0-4e1f-aa7f-95b12cfddf7e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:38:56.138998 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c05ea4-cec0-4e1f-aa7f-95b12cfddf7e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:38:56.140577 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c05ea4-cec0-4e1f-aa7f-95b12cfddf7e service cinder] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:38:56.161960 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1393/2781] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Jun 15 10:38:55 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 10:38:57.642743 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ca4bbb-4b1e-4f52-9b4e-c26c7d1f0f55 None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:38:57.643616 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ca4bbb-4b1e-4f52-9b4e-c26c7d1f0f55 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:38:57.644164 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ca4bbb-4b1e-4f52-9b4e-c26c7d1f0f55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:38:57.705600 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-11ca4bbb-4b1e-4f52-9b4e-c26c7d1f0f55 None None] MFA Rules not processed for user `07276de63b65403585d0d7c576447f96`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:38:57.711546 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-11ca4bbb-4b1e-4f52-9b4e-c26c7d1f0f55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:38:57.736964 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1389/2782] 10.222.0.49 () {58 vars in 957 bytes} [Wed Jun 15 10:38:57 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:38:57.879695 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c05ea4-cec0-4e1f-aa7f-95b12cfddf7e service cinder] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:38:57.881694 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-b3c05ea4-cec0-4e1f-aa7f-95b12cfddf7e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:38:57.906798 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c05ea4-cec0-4e1f-aa7f-95b12cfddf7e service cinder] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:38:57.924389 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6836bdf-87a4-423e-b805-78083f93b6a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:38:57.926052 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6836bdf-87a4-423e-b805-78083f93b6a2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:38:57.926763 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6836bdf-87a4-423e-b805-78083f93b6a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:38:57.927524 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6836bdf-87a4-423e-b805-78083f93b6a2 admin admin] PATH_INFO: `/v3/users/5d30c6b3a0234be39a7f24929d0ba1c2` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:38:57.929130 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6836bdf-87a4-423e-b805-78083f93b6a2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5d30c6b3a0234be39a7f24929d0ba1c2)` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:38:57.931340 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6836bdf-87a4-423e-b805-78083f93b6a2 admin admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:38:58.022095 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-f6836bdf-87a4-423e-b805-78083f93b6a2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5d30c6b3a0234be39a7f24929d0ba1c2'} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:38:58.032853 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-f6836bdf-87a4-423e-b805-78083f93b6a2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5d30c6b3a0234be39a7f24929d0ba1c2'} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:38:58.034244 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-f6836bdf-87a4-423e-b805-78083f93b6a2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5d30c6b3a0234be39a7f24929d0ba1c2'} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:38:58.035145 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-f6836bdf-87a4-423e-b805-78083f93b6a2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5d30c6b3a0234be39a7f24929d0ba1c2'} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:38:58.044271 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-f6836bdf-87a4-423e-b805-78083f93b6a2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5d30c6b3a0234be39a7f24929d0ba1c2'} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:38:58.046051 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:38:58.079036 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1394/2783] 10.222.0.49 () {58 vars in 1223 bytes} [Wed Jun 15 10:38:57 2022] DELETE /identity/v3/users/5d30c6b3a0234be39a7f24929d0ba1c2 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:38:58.083729 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aec96c0-a59c-4d68-a7d7-1c350c73ad4d None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:38:58.084223 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aec96c0-a59c-4d68-a7d7-1c350c73ad4d None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:38:58.084891 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aec96c0-a59c-4d68-a7d7-1c350c73ad4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:38:58.112408 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-9aec96c0-a59c-4d68-a7d7-1c350c73ad4d None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:38:58.114143 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-9aec96c0-a59c-4d68-a7d7-1c350c73ad4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:38:58.194472 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1390/2784] 10.222.0.49 () {58 vars in 957 bytes} [Wed Jun 15 10:38:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:38:58.204439 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6836bdf-87a4-423e-b805-78083f93b6a2 admin admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:38:58.206071 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-f6836bdf-87a4-423e-b805-78083f93b6a2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:38:58.224138 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aec96c0-a59c-4d68-a7d7-1c350c73ad4d None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:38:58.227816 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-9aec96c0-a59c-4d68-a7d7-1c350c73ad4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:38:58.317822 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6836bdf-87a4-423e-b805-78083f93b6a2 admin admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:38:58.322333 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613b516d-6eda-4255-bf10-cb4b1e023472 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0cb44a17d205431793e9a8791e477a32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '559b3e713f5844b6ae37151d56dd419f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:38:58.323034 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613b516d-6eda-4255-bf10-cb4b1e023472 service neutron] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:38:58.323522 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613b516d-6eda-4255-bf10-cb4b1e023472 service neutron] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:38:58.323812 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613b516d-6eda-4255-bf10-cb4b1e023472 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:38:58.344528 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-613b516d-6eda-4255-bf10-cb4b1e023472 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:38:58.346477 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-613b516d-6eda-4255-bf10-cb4b1e023472 service neutron] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:38:58.349921 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aec96c0-a59c-4d68-a7d7-1c350c73ad4d None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:38:58.355103 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0437ae-9f50-455c-a513-062fd6064623 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0cb44a17d205431793e9a8791e477a32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '559b3e713f5844b6ae37151d56dd419f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:38:58.355943 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0437ae-9f50-455c-a513-062fd6064623 service neutron] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:38:58.356228 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0437ae-9f50-455c-a513-062fd6064623 service neutron] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:38:58.356557 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0437ae-9f50-455c-a513-062fd6064623 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:38:58.358441 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1395/2785] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Jun 15 10:38:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:38:58.358586 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-5a0437ae-9f50-455c-a513-062fd6064623 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:38:58.370540 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a0437ae-9f50-455c-a513-062fd6064623 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:38:58.372107 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a0437ae-9f50-455c-a513-062fd6064623 service neutron] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:38:58.387018 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1391/2786] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Jun 15 10:38:58 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:38:58.731661 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613b516d-6eda-4255-bf10-cb4b1e023472 service neutron] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:38:58.737746 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-613b516d-6eda-4255-bf10-cb4b1e023472 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:38:58.753473 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613b516d-6eda-4255-bf10-cb4b1e023472 service neutron] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:38:58.780284 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b0233a-57cc-469d-bee6-7c4692a0b082 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:38:58.781738 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b0233a-57cc-469d-bee6-7c4692a0b082 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:38:58.782042 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b0233a-57cc-469d-bee6-7c4692a0b082 admin admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:38:58.782307 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b0233a-57cc-469d-bee6-7c4692a0b082 admin admin] PATH_INFO: `/v3/projects/36a6fb075f3b434d9ff5c7b0c68b85f1` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:38:58.784158 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b0233a-57cc-469d-bee6-7c4692a0b082 admin admin] RBAC: Authorizing `identity:delete_project(project_id=36a6fb075f3b434d9ff5c7b0c68b85f1)` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:38:58.786475 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b0233a-57cc-469d-bee6-7c4692a0b082 admin admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:38:58.867374 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-f4b0233a-57cc-469d-bee6-7c4692a0b082 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '36a6fb075f3b434d9ff5c7b0c68b85f1', 'request_id': 'req-f4b0233a-57cc-469d-bee6-7c4692a0b082'} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:38:58.883889 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-f4b0233a-57cc-469d-bee6-7c4692a0b082 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '36a6fb075f3b434d9ff5c7b0c68b85f1', 'request_id': 'req-f4b0233a-57cc-469d-bee6-7c4692a0b082'} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:38:58.893316 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-f4b0233a-57cc-469d-bee6-7c4692a0b082 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '36a6fb075f3b434d9ff5c7b0c68b85f1', 'request_id': 'req-f4b0233a-57cc-469d-bee6-7c4692a0b082'} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:38:58.945264 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-f4b0233a-57cc-469d-bee6-7c4692a0b082 admin admin] The token cache is being invalidate because project 36a6fb075f3b434d9ff5c7b0c68b85f1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80119) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 10:38:58.945569 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-f4b0233a-57cc-469d-bee6-7c4692a0b082 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:38:58.946258 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-f4b0233a-57cc-469d-bee6-7c4692a0b082 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:38:58.948005 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1396/2787] 10.222.0.49 () {58 vars in 1232 bytes} [Wed Jun 15 10:38:58 2022] DELETE /identity/v3/projects/36a6fb075f3b434d9ff5c7b0c68b85f1 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:38:58.952367 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0437ae-9f50-455c-a513-062fd6064623 service neutron] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:38:58.975395 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0437ae-9f50-455c-a513-062fd6064623 service neutron] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:38:59.048767 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:38:59.070882 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce1705d-2411-491c-b11e-be20a93122de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:38:59.071894 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce1705d-2411-491c-b11e-be20a93122de admin admin] REQUEST_METHOD: `DELETE` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:38:59.072256 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce1705d-2411-491c-b11e-be20a93122de admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:38:59.072613 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce1705d-2411-491c-b11e-be20a93122de admin admin] PATH_INFO: `/v3/users/11a093c78dda47f8b4b707761121ef65` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:38:59.074104 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce1705d-2411-491c-b11e-be20a93122de admin admin] RBAC: Authorizing `identity:delete_user(user_id=11a093c78dda47f8b4b707761121ef65)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:38:59.076265 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce1705d-2411-491c-b11e-be20a93122de admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:38:59.179408 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-9ce1705d-2411-491c-b11e-be20a93122de admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '11a093c78dda47f8b4b707761121ef65'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:38:59.187647 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-9ce1705d-2411-491c-b11e-be20a93122de admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '11a093c78dda47f8b4b707761121ef65'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:38:59.188596 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-9ce1705d-2411-491c-b11e-be20a93122de admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '11a093c78dda47f8b4b707761121ef65'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:38:59.189379 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-9ce1705d-2411-491c-b11e-be20a93122de admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '11a093c78dda47f8b4b707761121ef65'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:38:59.199356 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-9ce1705d-2411-491c-b11e-be20a93122de admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '11a093c78dda47f8b4b707761121ef65'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:38:59.226570 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1392/2788] 10.222.0.49 () {58 vars in 1223 bytes} [Wed Jun 15 10:38:58 2022] DELETE /identity/v3/users/11a093c78dda47f8b4b707761121ef65 => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:38:59.636353 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b0233a-57cc-469d-bee6-7c4692a0b082 admin admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:38:59.638791 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-f4b0233a-57cc-469d-bee6-7c4692a0b082 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:38:59.662632 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b0233a-57cc-469d-bee6-7c4692a0b082 admin admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:38:59.756047 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56cfa5b6-c716-4c06-b389-bc76ebd2e431 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:38:59.757462 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cfa5b6-c716-4c06-b389-bc76ebd2e431 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:38:59.759289 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cfa5b6-c716-4c06-b389-bc76ebd2e431 admin admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:38:59.759739 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cfa5b6-c716-4c06-b389-bc76ebd2e431 admin admin] PATH_INFO: `/v3/projects/776e1310de8b40a9819d42b46476c9f2` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:38:59.761144 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56cfa5b6-c716-4c06-b389-bc76ebd2e431 admin admin] RBAC: Authorizing `identity:delete_project(project_id=776e1310de8b40a9819d42b46476c9f2)` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:38:59.763525 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56cfa5b6-c716-4c06-b389-bc76ebd2e431 admin admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:38:59.785530 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:38:59.850243 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-56cfa5b6-c716-4c06-b389-bc76ebd2e431 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '776e1310de8b40a9819d42b46476c9f2', 'request_id': 'req-56cfa5b6-c716-4c06-b389-bc76ebd2e431'} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:38:59.863174 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-56cfa5b6-c716-4c06-b389-bc76ebd2e431 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '776e1310de8b40a9819d42b46476c9f2', 'request_id': 'req-56cfa5b6-c716-4c06-b389-bc76ebd2e431'} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:38:59.870318 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-56cfa5b6-c716-4c06-b389-bc76ebd2e431 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '776e1310de8b40a9819d42b46476c9f2', 'request_id': 'req-56cfa5b6-c716-4c06-b389-bc76ebd2e431'} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:38:59.883873 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-56cfa5b6-c716-4c06-b389-bc76ebd2e431 admin admin] The token cache is being invalidate because project 776e1310de8b40a9819d42b46476c9f2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80119) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 10:38:59.884151 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-56cfa5b6-c716-4c06-b389-bc76ebd2e431 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:38:59.885374 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-56cfa5b6-c716-4c06-b389-bc76ebd2e431 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:38:59.887252 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1397/2789] 10.222.0.49 () {58 vars in 1232 bytes} [Wed Jun 15 10:38:59 2022] DELETE /identity/v3/projects/776e1310de8b40a9819d42b46476c9f2 => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:38:59.891436 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce1705d-2411-491c-b11e-be20a93122de admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:38:59.893424 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-9ce1705d-2411-491c-b11e-be20a93122de admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:38:59.917184 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce1705d-2411-491c-b11e-be20a93122de admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:39:00.065463 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e9fea1-3a92-4f8a-9eee-bcd13631ca39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:39:00.067290 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e9fea1-3a92-4f8a-9eee-bcd13631ca39 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:39:00.067846 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e9fea1-3a92-4f8a-9eee-bcd13631ca39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:39:00.068926 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e9fea1-3a92-4f8a-9eee-bcd13631ca39 admin admin] PATH_INFO: `/v3/users/c786e2e9feec43d5b57ce496d7074121` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:39:00.072571 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e9fea1-3a92-4f8a-9eee-bcd13631ca39 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c786e2e9feec43d5b57ce496d7074121)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:39:00.078650 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e9fea1-3a92-4f8a-9eee-bcd13631ca39 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:39:00.183377 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-30e9fea1-3a92-4f8a-9eee-bcd13631ca39 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c786e2e9feec43d5b57ce496d7074121'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:39:00.200696 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-30e9fea1-3a92-4f8a-9eee-bcd13631ca39 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c786e2e9feec43d5b57ce496d7074121'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:39:00.201788 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-30e9fea1-3a92-4f8a-9eee-bcd13631ca39 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c786e2e9feec43d5b57ce496d7074121'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:39:00.204518 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-30e9fea1-3a92-4f8a-9eee-bcd13631ca39 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c786e2e9feec43d5b57ce496d7074121'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:39:00.217423 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-30e9fea1-3a92-4f8a-9eee-bcd13631ca39 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c786e2e9feec43d5b57ce496d7074121'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:39:00.220137 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=80,keystone:DELETE=18,keystone:INSERT=2 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:39:00.246587 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1393/2790] 10.222.0.49 () {58 vars in 1223 bytes} [Wed Jun 15 10:38:59 2022] DELETE /identity/v3/users/c786e2e9feec43d5b57ce496d7074121 => generated 0 bytes in 355 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:39:00.729836 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56cfa5b6-c716-4c06-b389-bc76ebd2e431 admin admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:39:00.731381 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-56cfa5b6-c716-4c06-b389-bc76ebd2e431 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:39:00.750340 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56cfa5b6-c716-4c06-b389-bc76ebd2e431 admin admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:39:00.819881 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79482cd5-98d4-4664-a989-b5c59bda7501 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:39:00.820675 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79482cd5-98d4-4664-a989-b5c59bda7501 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:39:00.820963 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79482cd5-98d4-4664-a989-b5c59bda7501 admin admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:39:00.821234 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79482cd5-98d4-4664-a989-b5c59bda7501 admin admin] PATH_INFO: `/v3/projects/d96dae0df643424590cd7b65167f3b87` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:39:00.822424 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79482cd5-98d4-4664-a989-b5c59bda7501 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d96dae0df643424590cd7b65167f3b87)` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:39:00.824370 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79482cd5-98d4-4664-a989-b5c59bda7501 admin admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:39:00.872822 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-79482cd5-98d4-4664-a989-b5c59bda7501 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd96dae0df643424590cd7b65167f3b87', 'request_id': 'req-79482cd5-98d4-4664-a989-b5c59bda7501'} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:39:00.881700 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-79482cd5-98d4-4664-a989-b5c59bda7501 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd96dae0df643424590cd7b65167f3b87', 'request_id': 'req-79482cd5-98d4-4664-a989-b5c59bda7501'} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:39:00.888207 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-79482cd5-98d4-4664-a989-b5c59bda7501 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd96dae0df643424590cd7b65167f3b87', 'request_id': 'req-79482cd5-98d4-4664-a989-b5c59bda7501'} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:39:00.899655 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-79482cd5-98d4-4664-a989-b5c59bda7501 admin admin] The token cache is being invalidate because project d96dae0df643424590cd7b65167f3b87 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80119) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 10:39:00.899948 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-79482cd5-98d4-4664-a989-b5c59bda7501 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:39:00.900705 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-79482cd5-98d4-4664-a989-b5c59bda7501 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:39:00.902483 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1398/2791] 10.222.0.49 () {58 vars in 1232 bytes} [Wed Jun 15 10:39:00 2022] DELETE /identity/v3/projects/d96dae0df643424590cd7b65167f3b87 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:39:00.909137 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de61075-8554-4dbf-bb4f-e7c792522e0f None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:39:00.909519 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de61075-8554-4dbf-bb4f-e7c792522e0f None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:39:00.909881 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de61075-8554-4dbf-bb4f-e7c792522e0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:39:00.929154 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-5de61075-8554-4dbf-bb4f-e7c792522e0f None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:39:00.930465 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-5de61075-8554-4dbf-bb4f-e7c792522e0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:39:01.003638 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1394/2792] 10.222.0.49 () {58 vars in 957 bytes} [Wed Jun 15 10:39:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:39:01.013155 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a413f44b-6bfc-4363-b9bf-940bb759ce41 None None] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:39:01.013445 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a413f44b-6bfc-4363-b9bf-940bb759ce41 None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:39:01.013723 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a413f44b-6bfc-4363-b9bf-940bb759ce41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:39:01.037384 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.auth.core [None req-a413f44b-6bfc-4363-b9bf-940bb759ce41 None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:39:01.038631 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-a413f44b-6bfc-4363-b9bf-940bb759ce41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:39:01.047876 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1399/2793] 10.222.0.49 () {58 vars in 957 bytes} [Wed Jun 15 10:39:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:39:01.051770 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de61075-8554-4dbf-bb4f-e7c792522e0f None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:39:01.069323 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de61075-8554-4dbf-bb4f-e7c792522e0f None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:39:01.074947 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6357706-ad57-48d8-9d32-7a256d4a05f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:39:01.075865 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6357706-ad57-48d8-9d32-7a256d4a05f0 admin admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:39:01.076155 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6357706-ad57-48d8-9d32-7a256d4a05f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:39:01.076430 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6357706-ad57-48d8-9d32-7a256d4a05f0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:39:01.077160 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6357706-ad57-48d8-9d32-7a256d4a05f0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:39:01.077463 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6357706-ad57-48d8-9d32-7a256d4a05f0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:39:01.079661 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6357706-ad57-48d8-9d32-7a256d4a05f0 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:39:01.103105 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1395/2794] 10.222.0.49 () {58 vars in 1152 bytes} [Wed Jun 15 10:39:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:39:01.107554 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a413f44b-6bfc-4363-b9bf-940bb759ce41 None None] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:39:01.118092 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a413f44b-6bfc-4363-b9bf-940bb759ce41 None None] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:39:01.123208 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c954f1-3739-4304-9528-dcf584e463cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:39:01.123881 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c954f1-3739-4304-9528-dcf584e463cf admin admin] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:39:01.124170 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c954f1-3739-4304-9528-dcf584e463cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:39:01.124435 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c954f1-3739-4304-9528-dcf584e463cf admin admin] PATH_INFO: `/v3/projects` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:39:01.125352 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c954f1-3739-4304-9528-dcf584e463cf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:39:01.127280 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c954f1-3739-4304-9528-dcf584e463cf admin admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:39:01.148614 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:39:01.162825 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1400/2795] 10.222.0.49 () {60 vars in 1152 bytes} [Wed Jun 15 10:39:01 2022] POST /identity/v3/projects => generated 373 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 10:39:01.165474 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6357706-ad57-48d8-9d32-7a256d4a05f0 admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:39:01.183588 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6357706-ad57-48d8-9d32-7a256d4a05f0 admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:39:01.254802 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10bf784-8ede-44ed-bd62-5edfb4a59c45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:39:01.255539 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10bf784-8ede-44ed-bd62-5edfb4a59c45 admin admin] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:39:01.255829 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10bf784-8ede-44ed-bd62-5edfb4a59c45 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:39:01.256109 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10bf784-8ede-44ed-bd62-5edfb4a59c45 admin admin] PATH_INFO: `/v3/users` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:39:01.256822 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c10bf784-8ede-44ed-bd62-5edfb4a59c45 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:39:01.258747 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c10bf784-8ede-44ed-bd62-5edfb4a59c45 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:39:01.283898 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1396/2796] 10.222.0.49 () {60 vars in 1143 bytes} [Wed Jun 15 10:39:01 2022] POST /identity/v3/users => generated 287 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 10:39:01.285170 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c954f1-3739-4304-9528-dcf584e463cf admin admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:39:01.295836 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c954f1-3739-4304-9528-dcf584e463cf admin admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:39:01.301452 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a214ab7-5ba3-4af2-8bac-f3897649c4cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:39:01.302038 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a214ab7-5ba3-4af2-8bac-f3897649c4cc admin admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:39:01.302436 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a214ab7-5ba3-4af2-8bac-f3897649c4cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:39:01.302598 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a214ab7-5ba3-4af2-8bac-f3897649c4cc admin admin] PATH_INFO: `/v3/roles` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:39:01.303565 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a214ab7-5ba3-4af2-8bac-f3897649c4cc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:39:01.305226 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a214ab7-5ba3-4af2-8bac-f3897649c4cc admin admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:39:01.319144 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1401/2797] 10.222.0.49 () {58 vars in 1121 bytes} [Wed Jun 15 10:39:01 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 10:39:01.329559 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10bf784-8ede-44ed-bd62-5edfb4a59c45 admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:39:01.334553 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10bf784-8ede-44ed-bd62-5edfb4a59c45 admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:39:01.339447 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3f9d2d-598e-4179-9d44-b2b0766b0c87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:39:01.340117 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3f9d2d-598e-4179-9d44-b2b0766b0c87 admin admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:39:01.340399 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3f9d2d-598e-4179-9d44-b2b0766b0c87 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:39:01.340678 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3f9d2d-598e-4179-9d44-b2b0766b0c87 admin admin] PATH_INFO: `/v3/roles` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:39:01.341406 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f3f9d2d-598e-4179-9d44-b2b0766b0c87 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:39:01.342980 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f3f9d2d-598e-4179-9d44-b2b0766b0c87 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:39:01.353260 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1397/2798] 10.222.0.49 () {58 vars in 1121 bytes} [Wed Jun 15 10:39:01 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 10:39:01.357614 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a214ab7-5ba3-4af2-8bac-f3897649c4cc admin admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:39:01.373996 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a214ab7-5ba3-4af2-8bac-f3897649c4cc admin admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:39:01.379470 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779c773a-8b0d-4202-886d-71bb6b41fd51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:39:01.380431 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779c773a-8b0d-4202-886d-71bb6b41fd51 admin admin] REQUEST_METHOD: `PUT` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:39:01.380715 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779c773a-8b0d-4202-886d-71bb6b41fd51 admin admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:39:01.380988 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779c773a-8b0d-4202-886d-71bb6b41fd51 admin admin] PATH_INFO: `/v3/projects/1f8d83e9e6a748cca6264ce1cc674ada/users/2dbb726151ae43939fb648ccadcab6d0/roles/6876e57cc51c448097ab109d35b2a3da` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:39:01.396458 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-779c773a-8b0d-4202-886d-71bb6b41fd51 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1f8d83e9e6a748cca6264ce1cc674ada, user_id=2dbb726151ae43939fb648ccadcab6d0, role_id=6876e57cc51c448097ab109d35b2a3da)` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:39:01.399422 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-779c773a-8b0d-4202-886d-71bb6b41fd51 admin admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:39:01.412730 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1402/2799] 10.222.0.49 () {60 vars in 1482 bytes} [Wed Jun 15 10:39:01 2022] PUT /identity/v3/projects/1f8d83e9e6a748cca6264ce1cc674ada/users/2dbb726151ae43939fb648ccadcab6d0/roles/6876e57cc51c448097ab109d35b2a3da => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:39:01.425585 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7593ea00-3381-4a08-879b-ffc67e7b682b None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:39:01.426049 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7593ea00-3381-4a08-879b-ffc67e7b682b None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:39:01.426328 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7593ea00-3381-4a08-879b-ffc67e7b682b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:39:01.461980 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-7593ea00-3381-4a08-879b-ffc67e7b682b None None] MFA Rules not processed for user `2dbb726151ae43939fb648ccadcab6d0`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:39:01.463259 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-7593ea00-3381-4a08-879b-ffc67e7b682b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:39:01.508411 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:39:01.541443 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1398/2800] 10.222.0.49 () {58 vars in 957 bytes} [Wed Jun 15 10:39:01 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:39:01.553331 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779c773a-8b0d-4202-886d-71bb6b41fd51 admin admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:39:01.555216 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-779c773a-8b0d-4202-886d-71bb6b41fd51 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:39:01.612600 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779c773a-8b0d-4202-886d-71bb6b41fd51 admin admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:39:01.616275 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ea1473-9d33-449d-b4de-315e2bcd5858 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20d1327553404b80aecee87c78fcb52b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '559b3e713f5844b6ae37151d56dd419f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:39:01.616929 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ea1473-9d33-449d-b4de-315e2bcd5858 service cinder] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:39:01.617216 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ea1473-9d33-449d-b4de-315e2bcd5858 service cinder] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:39:01.617491 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ea1473-9d33-449d-b4de-315e2bcd5858 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:39:01.627228 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ea1473-9d33-449d-b4de-315e2bcd5858 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:39:01.628694 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ea1473-9d33-449d-b4de-315e2bcd5858 service cinder] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:39:01.637385 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1403/2801] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Jun 15 10:39:01 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:39:01.663385 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7593ea00-3381-4a08-879b-ffc67e7b682b None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:39:01.726437 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7593ea00-3381-4a08-879b-ffc67e7b682b None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:39:01.731289 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2a016e-5055-4c1c-a0a7-28543c45ea1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:39:01.732521 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2a016e-5055-4c1c-a0a7-28543c45ea1d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:39:01.732804 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2a016e-5055-4c1c-a0a7-28543c45ea1d admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:39:01.733079 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2a016e-5055-4c1c-a0a7-28543c45ea1d admin admin] PATH_INFO: `/v3/users/2dbb726151ae43939fb648ccadcab6d0` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:39:01.734988 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff2a016e-5055-4c1c-a0a7-28543c45ea1d admin admin] RBAC: Authorizing `identity:delete_user(user_id=2dbb726151ae43939fb648ccadcab6d0)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:39:01.737334 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff2a016e-5055-4c1c-a0a7-28543c45ea1d admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:39:01.784461 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-ff2a016e-5055-4c1c-a0a7-28543c45ea1d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2dbb726151ae43939fb648ccadcab6d0'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:39:01.791666 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-ff2a016e-5055-4c1c-a0a7-28543c45ea1d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2dbb726151ae43939fb648ccadcab6d0'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:39:01.792466 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-ff2a016e-5055-4c1c-a0a7-28543c45ea1d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2dbb726151ae43939fb648ccadcab6d0'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:39:01.793116 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-ff2a016e-5055-4c1c-a0a7-28543c45ea1d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2dbb726151ae43939fb648ccadcab6d0'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:39:01.802031 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-ff2a016e-5055-4c1c-a0a7-28543c45ea1d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2dbb726151ae43939fb648ccadcab6d0'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:39:01.820842 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1399/2802] 10.222.0.49 () {58 vars in 1223 bytes} [Wed Jun 15 10:39:01 2022] DELETE /identity/v3/users/2dbb726151ae43939fb648ccadcab6d0 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:39:01.825354 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0428da76-ae89-46cc-97ea-4130f284f65c None None] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:39:01.825650 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0428da76-ae89-46cc-97ea-4130f284f65c None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:39:01.825933 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0428da76-ae89-46cc-97ea-4130f284f65c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:39:01.842539 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.auth.core [None req-0428da76-ae89-46cc-97ea-4130f284f65c None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:39:01.843817 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-0428da76-ae89-46cc-97ea-4130f284f65c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:39:01.903056 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1404/2803] 10.222.0.49 () {58 vars in 957 bytes} [Wed Jun 15 10:39:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:39:01.913820 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2a016e-5055-4c1c-a0a7-28543c45ea1d admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:39:01.915329 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-ff2a016e-5055-4c1c-a0a7-28543c45ea1d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:39:01.979577 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2a016e-5055-4c1c-a0a7-28543c45ea1d admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:39:01.983314 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7781c21-a53b-41fd-9377-cda520a3ec68 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0cb44a17d205431793e9a8791e477a32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '559b3e713f5844b6ae37151d56dd419f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:39:01.983969 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7781c21-a53b-41fd-9377-cda520a3ec68 service neutron] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:39:01.984746 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7781c21-a53b-41fd-9377-cda520a3ec68 service neutron] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:39:01.985025 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7781c21-a53b-41fd-9377-cda520a3ec68 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:39:01.994383 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7781c21-a53b-41fd-9377-cda520a3ec68 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:39:01.995848 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7781c21-a53b-41fd-9377-cda520a3ec68 service neutron] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:39:02.000330 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:39:02.007655 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1400/2804] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Jun 15 10:39:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 10:39:03.043788 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0428da76-ae89-46cc-97ea-4130f284f65c None None] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:39:03.045309 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-0428da76-ae89-46cc-97ea-4130f284f65c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:39:03.058919 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0428da76-ae89-46cc-97ea-4130f284f65c None None] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:39:03.064665 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e08d4d7-4f0c-4f71-a316-ce4b56e628a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:39:03.065569 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e08d4d7-4f0c-4f71-a316-ce4b56e628a0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:39:03.065852 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e08d4d7-4f0c-4f71-a316-ce4b56e628a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:39:03.066128 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e08d4d7-4f0c-4f71-a316-ce4b56e628a0 admin admin] PATH_INFO: `/v3/projects/1f8d83e9e6a748cca6264ce1cc674ada` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:39:03.067446 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e08d4d7-4f0c-4f71-a316-ce4b56e628a0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1f8d83e9e6a748cca6264ce1cc674ada)` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:39:03.069369 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e08d4d7-4f0c-4f71-a316-ce4b56e628a0 admin admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:39:03.128861 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-8e08d4d7-4f0c-4f71-a316-ce4b56e628a0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1f8d83e9e6a748cca6264ce1cc674ada', 'request_id': 'req-8e08d4d7-4f0c-4f71-a316-ce4b56e628a0'} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:39:03.138449 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-8e08d4d7-4f0c-4f71-a316-ce4b56e628a0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1f8d83e9e6a748cca6264ce1cc674ada', 'request_id': 'req-8e08d4d7-4f0c-4f71-a316-ce4b56e628a0'} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:39:03.143193 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=92,keystone:INSERT=2,keystone:DELETE=6 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:39:03.145943 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-8e08d4d7-4f0c-4f71-a316-ce4b56e628a0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1f8d83e9e6a748cca6264ce1cc674ada', 'request_id': 'req-8e08d4d7-4f0c-4f71-a316-ce4b56e628a0'} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:39:03.163874 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-8e08d4d7-4f0c-4f71-a316-ce4b56e628a0 admin admin] The token cache is being invalidate because project 1f8d83e9e6a748cca6264ce1cc674ada was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80119) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 10:39:03.164162 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-8e08d4d7-4f0c-4f71-a316-ce4b56e628a0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:39:03.165117 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-8e08d4d7-4f0c-4f71-a316-ce4b56e628a0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:39:03.166956 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1405/2805] 10.222.0.49 () {58 vars in 1232 bytes} [Wed Jun 15 10:39:03 2022] DELETE /identity/v3/projects/1f8d83e9e6a748cca6264ce1cc674ada => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:39:03.172595 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5abc76-22ca-46af-8b7d-65c5e63d4351 None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:39:03.173075 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5abc76-22ca-46af-8b7d-65c5e63d4351 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:39:03.173174 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5abc76-22ca-46af-8b7d-65c5e63d4351 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:39:03.196657 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-ad5abc76-22ca-46af-8b7d-65c5e63d4351 None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:39:03.197915 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-ad5abc76-22ca-46af-8b7d-65c5e63d4351 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:39:03.264215 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1401/2806] 10.222.0.49 () {58 vars in 957 bytes} [Wed Jun 15 10:39:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:39:03.272492 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51745a9-6651-4cdc-8b82-dd759028bd0a None None] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:39:03.272788 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51745a9-6651-4cdc-8b82-dd759028bd0a None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:39:03.273073 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51745a9-6651-4cdc-8b82-dd759028bd0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:39:03.295362 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.auth.core [None req-b51745a9-6651-4cdc-8b82-dd759028bd0a None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:39:03.296874 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-b51745a9-6651-4cdc-8b82-dd759028bd0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:39:03.307845 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1406/2807] 10.222.0.49 () {58 vars in 957 bytes} [Wed Jun 15 10:39:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:39:03.312235 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5abc76-22ca-46af-8b7d-65c5e63d4351 None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:39:03.322432 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5abc76-22ca-46af-8b7d-65c5e63d4351 None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:39:03.335945 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1560b4-ae1d-4f09-825c-f880f558c4f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:39:03.336648 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1560b4-ae1d-4f09-825c-f880f558c4f3 admin admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:39:03.336939 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1560b4-ae1d-4f09-825c-f880f558c4f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:39:03.337218 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1560b4-ae1d-4f09-825c-f880f558c4f3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:39:03.337972 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df1560b4-ae1d-4f09-825c-f880f558c4f3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:39:03.338289 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df1560b4-ae1d-4f09-825c-f880f558c4f3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:39:03.339955 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df1560b4-ae1d-4f09-825c-f880f558c4f3 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:39:03.358250 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1402/2808] 10.222.0.49 () {58 vars in 1152 bytes} [Wed Jun 15 10:39:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:39:03.369062 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51745a9-6651-4cdc-8b82-dd759028bd0a None None] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:39:03.387012 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51745a9-6651-4cdc-8b82-dd759028bd0a None None] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:39:03.392947 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bbbb852-b9ec-4c10-9e4e-4beb9900cc91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:39:03.393733 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbbb852-b9ec-4c10-9e4e-4beb9900cc91 admin admin] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:39:03.394103 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbbb852-b9ec-4c10-9e4e-4beb9900cc91 admin admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:39:03.394457 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbbb852-b9ec-4c10-9e4e-4beb9900cc91 admin admin] PATH_INFO: `/v3/projects` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:39:03.395262 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bbbb852-b9ec-4c10-9e4e-4beb9900cc91 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:39:03.397450 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bbbb852-b9ec-4c10-9e4e-4beb9900cc91 admin admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:39:03.428689 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1407/2809] 10.222.0.49 () {60 vars in 1152 bytes} [Wed Jun 15 10:39:03 2022] POST /identity/v3/projects => generated 381 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 10:39:03.432552 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1560b4-ae1d-4f09-825c-f880f558c4f3 admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:39:03.445722 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1560b4-ae1d-4f09-825c-f880f558c4f3 admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:39:03.528993 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779dd940-e25c-4f92-8910-debce8823978 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:39:03.529722 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779dd940-e25c-4f92-8910-debce8823978 admin admin] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:39:03.530089 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779dd940-e25c-4f92-8910-debce8823978 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:39:03.530374 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779dd940-e25c-4f92-8910-debce8823978 admin admin] PATH_INFO: `/v3/users` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:39:03.531828 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-779dd940-e25c-4f92-8910-debce8823978 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:39:03.533790 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-779dd940-e25c-4f92-8910-debce8823978 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:39:03.559971 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1403/2810] 10.222.0.49 () {60 vars in 1143 bytes} [Wed Jun 15 10:39:03 2022] POST /identity/v3/users => generated 291 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 10:39:03.564666 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bbbb852-b9ec-4c10-9e4e-4beb9900cc91 admin admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:39:03.586660 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bbbb852-b9ec-4c10-9e4e-4beb9900cc91 admin admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:39:03.599644 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfea7de8-7a4a-4cb6-8bf8-caac0bbb79c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:39:03.601680 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfea7de8-7a4a-4cb6-8bf8-caac0bbb79c1 admin admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:39:03.601975 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfea7de8-7a4a-4cb6-8bf8-caac0bbb79c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:39:03.603249 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfea7de8-7a4a-4cb6-8bf8-caac0bbb79c1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:39:03.605792 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfea7de8-7a4a-4cb6-8bf8-caac0bbb79c1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:39:03.608543 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfea7de8-7a4a-4cb6-8bf8-caac0bbb79c1 admin admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:39:03.623296 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1408/2811] 10.222.0.49 () {58 vars in 1121 bytes} [Wed Jun 15 10:39:03 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 15 10:39:03.628474 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779dd940-e25c-4f92-8910-debce8823978 admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:39:03.641231 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779dd940-e25c-4f92-8910-debce8823978 admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:39:03.646334 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa39a1e-634b-4e7a-b002-9a7501569d61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:39:03.647014 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa39a1e-634b-4e7a-b002-9a7501569d61 admin admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:39:03.647333 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa39a1e-634b-4e7a-b002-9a7501569d61 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:39:03.647661 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa39a1e-634b-4e7a-b002-9a7501569d61 admin admin] PATH_INFO: `/v3/roles` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:39:03.648405 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa39a1e-634b-4e7a-b002-9a7501569d61 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:39:03.650497 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa39a1e-634b-4e7a-b002-9a7501569d61 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:39:03.664570 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1404/2812] 10.222.0.49 () {58 vars in 1121 bytes} [Wed Jun 15 10:39:03 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 10:39:03.668554 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfea7de8-7a4a-4cb6-8bf8-caac0bbb79c1 admin admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:39:03.680109 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfea7de8-7a4a-4cb6-8bf8-caac0bbb79c1 admin admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:39:03.686231 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683fb014-e003-4b6c-8186-e629a2cc28d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:39:03.687844 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683fb014-e003-4b6c-8186-e629a2cc28d2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:39:03.688135 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683fb014-e003-4b6c-8186-e629a2cc28d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:39:03.688467 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683fb014-e003-4b6c-8186-e629a2cc28d2 admin admin] PATH_INFO: `/v3/projects/129b7faaaf264434baf88ffe0fe9bb25/users/46fc1f62b79541e7a550731ee4728637/roles/6876e57cc51c448097ab109d35b2a3da` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:39:03.705027 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-683fb014-e003-4b6c-8186-e629a2cc28d2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=129b7faaaf264434baf88ffe0fe9bb25, user_id=46fc1f62b79541e7a550731ee4728637, role_id=6876e57cc51c448097ab109d35b2a3da)` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:39:03.707986 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-683fb014-e003-4b6c-8186-e629a2cc28d2 admin admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:39:03.722863 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1409/2813] 10.222.0.49 () {60 vars in 1482 bytes} [Wed Jun 15 10:39:03 2022] PUT /identity/v3/projects/129b7faaaf264434baf88ffe0fe9bb25/users/46fc1f62b79541e7a550731ee4728637/roles/6876e57cc51c448097ab109d35b2a3da => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 10:39:03.727955 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294760b9-4565-45fb-8212-6a097ae9d404 None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:39:03.728252 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294760b9-4565-45fb-8212-6a097ae9d404 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:39:03.728531 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294760b9-4565-45fb-8212-6a097ae9d404 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:39:03.746364 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-294760b9-4565-45fb-8212-6a097ae9d404 None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:39:03.747617 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-294760b9-4565-45fb-8212-6a097ae9d404 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:39:03.814554 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1405/2814] 10.222.0.49 () {58 vars in 957 bytes} [Wed Jun 15 10:39:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:39:03.824286 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683fb014-e003-4b6c-8186-e629a2cc28d2 admin admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:39:03.826047 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-683fb014-e003-4b6c-8186-e629a2cc28d2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:39:03.890090 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683fb014-e003-4b6c-8186-e629a2cc28d2 admin admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:39:03.894422 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7d16c8-4e3e-4ebf-b1bc-7917f660a668 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0cb44a17d205431793e9a8791e477a32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '559b3e713f5844b6ae37151d56dd419f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:39:03.895079 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7d16c8-4e3e-4ebf-b1bc-7917f660a668 service neutron] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:39:03.895393 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7d16c8-4e3e-4ebf-b1bc-7917f660a668 service neutron] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:39:03.895672 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7d16c8-4e3e-4ebf-b1bc-7917f660a668 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:39:03.905252 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb7d16c8-4e3e-4ebf-b1bc-7917f660a668 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:39:03.906725 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb7d16c8-4e3e-4ebf-b1bc-7917f660a668 service neutron] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:39:03.916346 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1410/2815] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Jun 15 10:39:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 10:39:04.605736 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab8fc94-0240-46df-be9f-5cc2cde99935 None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:39:04.606513 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab8fc94-0240-46df-be9f-5cc2cde99935 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:39:04.608137 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab8fc94-0240-46df-be9f-5cc2cde99935 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:39:04.623521 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:39:04.649454 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-dab8fc94-0240-46df-be9f-5cc2cde99935 None None] MFA Rules not processed for user `46fc1f62b79541e7a550731ee4728637`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:39:04.650723 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-dab8fc94-0240-46df-be9f-5cc2cde99935 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:39:04.746527 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1406/2816] 10.222.0.49 () {58 vars in 957 bytes} [Wed Jun 15 10:39:04 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:39:04.755774 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7d16c8-4e3e-4ebf-b1bc-7917f660a668 service neutron] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:39:04.757303 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-cb7d16c8-4e3e-4ebf-b1bc-7917f660a668 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:39:04.820921 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7d16c8-4e3e-4ebf-b1bc-7917f660a668 service neutron] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:39:04.824553 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4191aa-418e-4ee1-9af0-dfdcb5fd8efe service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20d1327553404b80aecee87c78fcb52b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '559b3e713f5844b6ae37151d56dd419f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:39:04.825211 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4191aa-418e-4ee1-9af0-dfdcb5fd8efe service cinder] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:39:04.825496 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4191aa-418e-4ee1-9af0-dfdcb5fd8efe service cinder] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:39:04.825895 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4191aa-418e-4ee1-9af0-dfdcb5fd8efe service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:39:04.835081 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4191aa-418e-4ee1-9af0-dfdcb5fd8efe service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:39:04.836571 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4191aa-418e-4ee1-9af0-dfdcb5fd8efe service cinder] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:39:04.846444 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1411/2817] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Jun 15 10:39:04 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 10:39:06.250920 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a7144e-8f27-4676-bb98-05ff504732ca None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:39:06.251478 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a7144e-8f27-4676-bb98-05ff504732ca None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:39:06.251597 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a7144e-8f27-4676-bb98-05ff504732ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:39:06.270106 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-c0a7144e-8f27-4676-bb98-05ff504732ca None None] MFA Rules not processed for user `46fc1f62b79541e7a550731ee4728637`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:39:06.271350 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-c0a7144e-8f27-4676-bb98-05ff504732ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:39:06.278411 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1407/2818] 10.222.0.49 () {58 vars in 957 bytes} [Wed Jun 15 10:39:06 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:39:12.180959 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e39b2a-e792-4cd6-be4c-4b3bcf7305d3 None None] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:39:12.181487 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e39b2a-e792-4cd6-be4c-4b3bcf7305d3 None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:39:12.181603 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e39b2a-e792-4cd6-be4c-4b3bcf7305d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:39:12.199754 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.auth.core [None req-13e39b2a-e792-4cd6-be4c-4b3bcf7305d3 None None] MFA Rules not processed for user `61bc9ad2c9444fc297eb4b5902a67b98`. Rule list: `[]` (Enabled: `True`). {{(pid=80119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:39:12.200986 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-13e39b2a-e792-4cd6-be4c-4b3bcf7305d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:39:12.227394 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:DELETE=1,keystone:SELECT=97,keystone:INSERT=2 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:39:12.259730 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1412/2819] 10.222.0.49 () {58 vars in 957 bytes} [Wed Jun 15 10:39:12 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:39:14.422954 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da567cfe-0c7f-4057-af71-60d78b965d6d None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:39:14.423476 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da567cfe-0c7f-4057-af71-60d78b965d6d None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:39:14.423582 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da567cfe-0c7f-4057-af71-60d78b965d6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:39:14.442046 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-da567cfe-0c7f-4057-af71-60d78b965d6d None None] MFA Rules not processed for user `103e10244eca444b86b9c51ff314d445`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:39:14.443308 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-da567cfe-0c7f-4057-af71-60d78b965d6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:39:14.502961 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1408/2820] 10.222.0.49 () {58 vars in 1009 bytes} [Wed Jun 15 10:39:14 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:39:14.551502 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e39b2a-e792-4cd6-be4c-4b3bcf7305d3 None None] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:39:14.552932 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-13e39b2a-e792-4cd6-be4c-4b3bcf7305d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:39:14.637688 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e39b2a-e792-4cd6-be4c-4b3bcf7305d3 None None] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:39:14.643307 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb2e7fe-c3b9-4000-9ae1-5dfa95a5dd7f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a69f99210444f9696e8b22615fb4a41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '559b3e713f5844b6ae37151d56dd419f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:39:14.643966 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb2e7fe-c3b9-4000-9ae1-5dfa95a5dd7f service swift] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:39:14.644256 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb2e7fe-c3b9-4000-9ae1-5dfa95a5dd7f service swift] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:39:14.644542 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb2e7fe-c3b9-4000-9ae1-5dfa95a5dd7f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:39:14.655694 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb2e7fe-c3b9-4000-9ae1-5dfa95a5dd7f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:39:14.657878 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb2e7fe-c3b9-4000-9ae1-5dfa95a5dd7f service swift] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:39:14.666635 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1413/2821] 10.222.0.49 () {60 vars in 1384 bytes} [Wed Jun 15 10:39:14 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 15 10:39:15.178311 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b6c93b-ab02-4fe2-8ae1-7eb3f18a6244 None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:39:15.178627 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b6c93b-ab02-4fe2-8ae1-7eb3f18a6244 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:39:15.178901 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b6c93b-ab02-4fe2-8ae1-7eb3f18a6244 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:39:15.195984 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-b1b6c93b-ab02-4fe2-8ae1-7eb3f18a6244 None None] MFA Rules not processed for user `103e10244eca444b86b9c51ff314d445`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:39:15.197166 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-b1b6c93b-ab02-4fe2-8ae1-7eb3f18a6244 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:39:15.205220 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1409/2822] 10.222.0.49 () {58 vars in 967 bytes} [Wed Jun 15 10:39:15 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:39:15.219185 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb2e7fe-c3b9-4000-9ae1-5dfa95a5dd7f service swift] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:39:15.229563 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb2e7fe-c3b9-4000-9ae1-5dfa95a5dd7f service swift] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:39:15.234616 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e5c1ef-7f46-4686-853a-46397d611243 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a69f99210444f9696e8b22615fb4a41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '559b3e713f5844b6ae37151d56dd419f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:39:15.235318 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e5c1ef-7f46-4686-853a-46397d611243 service swift] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:39:15.235606 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e5c1ef-7f46-4686-853a-46397d611243 service swift] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:39:15.235911 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e5c1ef-7f46-4686-853a-46397d611243 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:39:15.244006 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e5c1ef-7f46-4686-853a-46397d611243 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:39:15.245621 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e5c1ef-7f46-4686-853a-46397d611243 service swift] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:39:15.253625 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1414/2823] 10.222.0.49 () {60 vars in 1384 bytes} [Wed Jun 15 10:39:15 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 15 10:39:15.441778 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5080055-4eeb-4088-9c53-796739d11758 None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:39:15.442265 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5080055-4eeb-4088-9c53-796739d11758 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:39:15.442405 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5080055-4eeb-4088-9c53-796739d11758 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:39:15.463007 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-c5080055-4eeb-4088-9c53-796739d11758 None None] MFA Rules not processed for user `103e10244eca444b86b9c51ff314d445`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:39:15.464289 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-c5080055-4eeb-4088-9c53-796739d11758 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:39:15.472150 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1410/2824] 10.222.0.49 () {58 vars in 1009 bytes} [Wed Jun 15 10:39:15 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:39:15.483913 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e5c1ef-7f46-4686-853a-46397d611243 service swift] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:39:15.494645 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e5c1ef-7f46-4686-853a-46397d611243 service swift] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:39:15.499784 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05ff401-f9bb-43b7-8ede-86c4c5cf108b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a69f99210444f9696e8b22615fb4a41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '559b3e713f5844b6ae37151d56dd419f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:39:15.500425 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05ff401-f9bb-43b7-8ede-86c4c5cf108b service swift] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:39:15.500713 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05ff401-f9bb-43b7-8ede-86c4c5cf108b service swift] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:39:15.500996 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05ff401-f9bb-43b7-8ede-86c4c5cf108b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:39:15.509089 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a05ff401-f9bb-43b7-8ede-86c4c5cf108b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:39:15.510670 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a05ff401-f9bb-43b7-8ede-86c4c5cf108b service swift] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:39:15.519466 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1415/2825] 10.222.0.49 () {60 vars in 1384 bytes} [Wed Jun 15 10:39:15 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 15 10:39:15.840711 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0155ee-00da-4e45-b92f-1a597a10a4b3 None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:39:15.841080 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0155ee-00da-4e45-b92f-1a597a10a4b3 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:39:15.841457 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0155ee-00da-4e45-b92f-1a597a10a4b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:39:15.860206 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-fa0155ee-00da-4e45-b92f-1a597a10a4b3 None None] MFA Rules not processed for user `103e10244eca444b86b9c51ff314d445`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:39:15.861527 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-fa0155ee-00da-4e45-b92f-1a597a10a4b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:39:15.868915 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1411/2826] 10.222.0.49 () {58 vars in 967 bytes} [Wed Jun 15 10:39:15 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:39:15.881864 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05ff401-f9bb-43b7-8ede-86c4c5cf108b service swift] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:39:15.892765 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05ff401-f9bb-43b7-8ede-86c4c5cf108b service swift] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:39:15.897964 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dcbde93-3948-476f-95be-29a1be20dd95 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a69f99210444f9696e8b22615fb4a41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '559b3e713f5844b6ae37151d56dd419f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:39:15.898890 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcbde93-3948-476f-95be-29a1be20dd95 service swift] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:39:15.899215 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcbde93-3948-476f-95be-29a1be20dd95 service swift] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:39:15.899504 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcbde93-3948-476f-95be-29a1be20dd95 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:39:15.908696 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dcbde93-3948-476f-95be-29a1be20dd95 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:39:15.910300 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dcbde93-3948-476f-95be-29a1be20dd95 service swift] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:39:15.918564 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1416/2827] 10.222.0.49 () {60 vars in 1384 bytes} [Wed Jun 15 10:39:15 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 15 10:39:18.130930 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0155ee-00da-4e45-b92f-1a597a10a4b3 None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:39:18.132940 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-fa0155ee-00da-4e45-b92f-1a597a10a4b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:39:18.147414 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0155ee-00da-4e45-b92f-1a597a10a4b3 None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:39:18.152514 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09da7952-fcbf-45c5-ac66-18ca90d26e74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:39:18.153360 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09da7952-fcbf-45c5-ac66-18ca90d26e74 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:39:18.153648 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09da7952-fcbf-45c5-ac66-18ca90d26e74 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:39:18.153929 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09da7952-fcbf-45c5-ac66-18ca90d26e74 admin admin] PATH_INFO: `/v3/users/8a8ea7b3022e44f48179326c25646c38` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:39:18.155212 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09da7952-fcbf-45c5-ac66-18ca90d26e74 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8a8ea7b3022e44f48179326c25646c38)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:39:18.157163 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09da7952-fcbf-45c5-ac66-18ca90d26e74 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:39:18.204428 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-09da7952-fcbf-45c5-ac66-18ca90d26e74 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8a8ea7b3022e44f48179326c25646c38'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:39:18.211198 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-09da7952-fcbf-45c5-ac66-18ca90d26e74 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8a8ea7b3022e44f48179326c25646c38'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:39:18.211869 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-09da7952-fcbf-45c5-ac66-18ca90d26e74 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8a8ea7b3022e44f48179326c25646c38'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:39:18.212512 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-09da7952-fcbf-45c5-ac66-18ca90d26e74 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8a8ea7b3022e44f48179326c25646c38'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:39:18.219896 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-09da7952-fcbf-45c5-ac66-18ca90d26e74 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8a8ea7b3022e44f48179326c25646c38'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:39:18.224648 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:39:18.240090 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1412/2828] 10.222.0.49 () {58 vars in 1223 bytes} [Wed Jun 15 10:39:18 2022] DELETE /identity/v3/users/8a8ea7b3022e44f48179326c25646c38 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:39:18.244618 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edd479d-83a7-4b14-aa8a-e99c3a7f32dd None None] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:39:18.245124 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edd479d-83a7-4b14-aa8a-e99c3a7f32dd None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:39:18.245247 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edd479d-83a7-4b14-aa8a-e99c3a7f32dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:39:18.263298 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.auth.core [None req-9edd479d-83a7-4b14-aa8a-e99c3a7f32dd None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:39:18.264529 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-9edd479d-83a7-4b14-aa8a-e99c3a7f32dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:39:18.328531 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1417/2829] 10.222.0.49 () {58 vars in 957 bytes} [Wed Jun 15 10:39:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 10:39:18.347385 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09da7952-fcbf-45c5-ac66-18ca90d26e74 admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:39:18.349191 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-09da7952-fcbf-45c5-ac66-18ca90d26e74 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:39:18.415781 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09da7952-fcbf-45c5-ac66-18ca90d26e74 admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:39:18.421122 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96bb8497-a973-4684-ac90-98ac9b5ca212 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0cb44a17d205431793e9a8791e477a32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '559b3e713f5844b6ae37151d56dd419f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:39:18.422488 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bb8497-a973-4684-ac90-98ac9b5ca212 service neutron] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:39:18.422783 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bb8497-a973-4684-ac90-98ac9b5ca212 service neutron] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:39:18.423067 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bb8497-a973-4684-ac90-98ac9b5ca212 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:39:18.438225 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96bb8497-a973-4684-ac90-98ac9b5ca212 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:39:18.439900 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96bb8497-a973-4684-ac90-98ac9b5ca212 service neutron] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:39:18.463187 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1413/2830] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Jun 15 10:39:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:39:18.555678 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990c1d2d-3208-44c6-8089-49d762b6fdf9 None None] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:39:18.555969 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990c1d2d-3208-44c6-8089-49d762b6fdf9 None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:39:18.556253 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990c1d2d-3208-44c6-8089-49d762b6fdf9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:39:18.574728 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.auth.core [None req-990c1d2d-3208-44c6-8089-49d762b6fdf9 None None] MFA Rules not processed for user `07276de63b65403585d0d7c576447f96`. Rule list: `[]` (Enabled: `True`). {{(pid=80119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:39:18.576301 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-990c1d2d-3208-44c6-8089-49d762b6fdf9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:39:18.600692 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=100 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:39:18.673632 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96bb8497-a973-4684-ac90-98ac9b5ca212 service neutron] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:39:18.677999 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-96bb8497-a973-4684-ac90-98ac9b5ca212 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:39:18.684527 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1418/2831] 10.222.0.49 () {58 vars in 957 bytes} [Wed Jun 15 10:39:18 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:39:18.703429 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96bb8497-a973-4684-ac90-98ac9b5ca212 service neutron] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:39:18.713991 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c834a5-31e6-4e2d-bc3b-578c755930d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:39:18.714873 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c834a5-31e6-4e2d-bc3b-578c755930d9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:39:18.715752 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c834a5-31e6-4e2d-bc3b-578c755930d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:39:18.716215 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c834a5-31e6-4e2d-bc3b-578c755930d9 admin admin] PATH_INFO: `/v3/projects/aab8980da8444a9789dd0c0b4b88aeac` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:39:18.717590 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c834a5-31e6-4e2d-bc3b-578c755930d9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=aab8980da8444a9789dd0c0b4b88aeac)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:39:18.724541 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c834a5-31e6-4e2d-bc3b-578c755930d9 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:39:18.828105 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-35c834a5-31e6-4e2d-bc3b-578c755930d9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aab8980da8444a9789dd0c0b4b88aeac', 'request_id': 'req-35c834a5-31e6-4e2d-bc3b-578c755930d9'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:39:18.838826 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-35c834a5-31e6-4e2d-bc3b-578c755930d9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aab8980da8444a9789dd0c0b4b88aeac', 'request_id': 'req-35c834a5-31e6-4e2d-bc3b-578c755930d9'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:39:18.848107 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-35c834a5-31e6-4e2d-bc3b-578c755930d9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aab8980da8444a9789dd0c0b4b88aeac', 'request_id': 'req-35c834a5-31e6-4e2d-bc3b-578c755930d9'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:39:18.869509 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-35c834a5-31e6-4e2d-bc3b-578c755930d9 admin admin] The token cache is being invalidate because project aab8980da8444a9789dd0c0b4b88aeac was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80120) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 10:39:18.870106 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-35c834a5-31e6-4e2d-bc3b-578c755930d9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:39:18.871252 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-35c834a5-31e6-4e2d-bc3b-578c755930d9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:39:18.877651 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1414/2832] 10.222.0.49 () {58 vars in 1232 bytes} [Wed Jun 15 10:39:18 2022] DELETE /identity/v3/projects/aab8980da8444a9789dd0c0b4b88aeac => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:39:18.882284 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756d27e4-2296-46c0-a53e-53278a3c7ed1 None None] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:39:18.882626 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756d27e4-2296-46c0-a53e-53278a3c7ed1 None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:39:18.882933 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756d27e4-2296-46c0-a53e-53278a3c7ed1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:39:18.910267 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.auth.core [None req-756d27e4-2296-46c0-a53e-53278a3c7ed1 None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:39:18.913470 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-756d27e4-2296-46c0-a53e-53278a3c7ed1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:39:19.017508 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1419/2833] 10.222.0.49 () {58 vars in 957 bytes} [Wed Jun 15 10:39:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:39:19.026505 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81114267-4829-4c95-b29f-1a66be7819a8 None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:39:19.026799 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81114267-4829-4c95-b29f-1a66be7819a8 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:39:19.027168 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81114267-4829-4c95-b29f-1a66be7819a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:39:19.052767 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-81114267-4829-4c95-b29f-1a66be7819a8 None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:39:19.054388 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-81114267-4829-4c95-b29f-1a66be7819a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:39:19.087272 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1415/2834] 10.222.0.49 () {58 vars in 957 bytes} [Wed Jun 15 10:39:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:39:19.087601 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756d27e4-2296-46c0-a53e-53278a3c7ed1 None None] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:39:19.102547 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756d27e4-2296-46c0-a53e-53278a3c7ed1 None None] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:39:19.124386 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745f431f-c8ec-4ebf-a4ad-99eb49fa12e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:39:19.126334 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745f431f-c8ec-4ebf-a4ad-99eb49fa12e1 admin admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:39:19.126880 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745f431f-c8ec-4ebf-a4ad-99eb49fa12e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:39:19.128396 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745f431f-c8ec-4ebf-a4ad-99eb49fa12e1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:39:19.129668 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-745f431f-c8ec-4ebf-a4ad-99eb49fa12e1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80119) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:39:19.130456 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-745f431f-c8ec-4ebf-a4ad-99eb49fa12e1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:39:19.134906 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-745f431f-c8ec-4ebf-a4ad-99eb49fa12e1 admin admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:39:19.156493 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81114267-4829-4c95-b29f-1a66be7819a8 None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:39:19.167631 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1420/2835] 10.222.0.49 () {58 vars in 1152 bytes} [Wed Jun 15 10:39:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 10:39:19.168587 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81114267-4829-4c95-b29f-1a66be7819a8 None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:39:19.175934 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95df4ab3-d2cb-49c9-9a28-0f0e2866b5da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:39:19.176866 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95df4ab3-d2cb-49c9-9a28-0f0e2866b5da admin admin] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:39:19.177367 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95df4ab3-d2cb-49c9-9a28-0f0e2866b5da admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:39:19.177936 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95df4ab3-d2cb-49c9-9a28-0f0e2866b5da admin admin] PATH_INFO: `/v3/projects` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:39:19.178868 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95df4ab3-d2cb-49c9-9a28-0f0e2866b5da admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:39:19.181210 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95df4ab3-d2cb-49c9-9a28-0f0e2866b5da admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:39:19.213580 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1416/2836] 10.222.0.49 () {60 vars in 1152 bytes} [Wed Jun 15 10:39:19 2022] POST /identity/v3/projects => generated 371 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 10:39:19.217577 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745f431f-c8ec-4ebf-a4ad-99eb49fa12e1 admin admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:39:19.229481 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745f431f-c8ec-4ebf-a4ad-99eb49fa12e1 admin admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:39:19.296596 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b85a90-beec-47e2-b456-00cf3a94a1e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:39:19.297552 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b85a90-beec-47e2-b456-00cf3a94a1e5 admin admin] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:39:19.297938 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b85a90-beec-47e2-b456-00cf3a94a1e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:39:19.298394 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b85a90-beec-47e2-b456-00cf3a94a1e5 admin admin] PATH_INFO: `/v3/users` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:39:19.299461 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b85a90-beec-47e2-b456-00cf3a94a1e5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:39:19.301565 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b85a90-beec-47e2-b456-00cf3a94a1e5 admin admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:39:19.330523 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1421/2837] 10.222.0.49 () {60 vars in 1143 bytes} [Wed Jun 15 10:39:19 2022] POST /identity/v3/users => generated 286 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 10:39:19.334777 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95df4ab3-d2cb-49c9-9a28-0f0e2866b5da admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:39:19.347383 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95df4ab3-d2cb-49c9-9a28-0f0e2866b5da admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:39:19.352341 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93263fb-f944-4c83-bc18-2345f5ba1b20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:39:19.353029 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93263fb-f944-4c83-bc18-2345f5ba1b20 admin admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:39:19.353352 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93263fb-f944-4c83-bc18-2345f5ba1b20 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:39:19.353632 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93263fb-f944-4c83-bc18-2345f5ba1b20 admin admin] PATH_INFO: `/v3/roles` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:39:19.354372 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f93263fb-f944-4c83-bc18-2345f5ba1b20 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:39:19.356114 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f93263fb-f944-4c83-bc18-2345f5ba1b20 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:39:19.370460 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1417/2838] 10.222.0.49 () {58 vars in 1121 bytes} [Wed Jun 15 10:39:19 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 10:39:19.372322 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b85a90-beec-47e2-b456-00cf3a94a1e5 admin admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:39:19.386220 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b85a90-beec-47e2-b456-00cf3a94a1e5 admin admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:39:19.392284 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97544b4-6d94-4c06-8e97-e20f2af09160 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:39:19.392980 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97544b4-6d94-4c06-8e97-e20f2af09160 admin admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:39:19.393266 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97544b4-6d94-4c06-8e97-e20f2af09160 admin admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:39:19.393545 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97544b4-6d94-4c06-8e97-e20f2af09160 admin admin] PATH_INFO: `/v3/roles` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:39:19.394347 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e97544b4-6d94-4c06-8e97-e20f2af09160 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:39:19.396608 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e97544b4-6d94-4c06-8e97-e20f2af09160 admin admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:39:19.408758 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1422/2839] 10.222.0.49 () {58 vars in 1121 bytes} [Wed Jun 15 10:39:19 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 10:39:19.412825 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93263fb-f944-4c83-bc18-2345f5ba1b20 admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:39:19.423300 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93263fb-f944-4c83-bc18-2345f5ba1b20 admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:39:19.427931 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f896b3f-237f-4b01-9427-55b8804a010c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:39:19.428837 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f896b3f-237f-4b01-9427-55b8804a010c admin admin] REQUEST_METHOD: `PUT` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:39:19.429146 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f896b3f-237f-4b01-9427-55b8804a010c admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:39:19.429450 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f896b3f-237f-4b01-9427-55b8804a010c admin admin] PATH_INFO: `/v3/projects/fe56b435fdbf49629bfc5e06c24e78b9/users/0b4d91a8096f4f70983ebad7e097a78a/roles/6876e57cc51c448097ab109d35b2a3da` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:39:19.445981 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f896b3f-237f-4b01-9427-55b8804a010c admin admin] RBAC: Authorizing `identity:create_grant(project_id=fe56b435fdbf49629bfc5e06c24e78b9, user_id=0b4d91a8096f4f70983ebad7e097a78a, role_id=6876e57cc51c448097ab109d35b2a3da)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:39:19.448721 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f896b3f-237f-4b01-9427-55b8804a010c admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:39:19.463154 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1418/2840] 10.222.0.49 () {60 vars in 1482 bytes} [Wed Jun 15 10:39:19 2022] PUT /identity/v3/projects/fe56b435fdbf49629bfc5e06c24e78b9/users/0b4d91a8096f4f70983ebad7e097a78a/roles/6876e57cc51c448097ab109d35b2a3da => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:39:19.468315 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67089437-509f-49d5-bcf2-4093edd5b15c None None] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:39:19.468610 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67089437-509f-49d5-bcf2-4093edd5b15c None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:39:19.468894 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67089437-509f-49d5-bcf2-4093edd5b15c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:39:19.488447 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.auth.core [None req-67089437-509f-49d5-bcf2-4093edd5b15c None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:39:19.490531 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-67089437-509f-49d5-bcf2-4093edd5b15c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:39:19.530429 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:39:19.559768 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1423/2841] 10.222.0.49 () {58 vars in 957 bytes} [Wed Jun 15 10:39:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:39:19.569773 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f896b3f-237f-4b01-9427-55b8804a010c admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:39:19.571576 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-4f896b3f-237f-4b01-9427-55b8804a010c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:39:19.582005 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:DELETE=9,keystone:SELECT=88,keystone:INSERT=3 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:39:19.636822 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f896b3f-237f-4b01-9427-55b8804a010c admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:39:19.641530 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017ace4b-e272-4280-aa0b-7e11d92dbc9f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0cb44a17d205431793e9a8791e477a32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '559b3e713f5844b6ae37151d56dd419f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:39:19.642270 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017ace4b-e272-4280-aa0b-7e11d92dbc9f service neutron] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:39:19.642632 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017ace4b-e272-4280-aa0b-7e11d92dbc9f service neutron] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:39:19.643025 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017ace4b-e272-4280-aa0b-7e11d92dbc9f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:39:19.655988 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-017ace4b-e272-4280-aa0b-7e11d92dbc9f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:39:19.657785 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-017ace4b-e272-4280-aa0b-7e11d92dbc9f service neutron] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:39:19.675469 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1419/2842] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Jun 15 10:39:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:39:25.213685 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33430a9d-975f-4c0d-9f96-8e3b9c31b72a None None] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:39:25.214178 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33430a9d-975f-4c0d-9f96-8e3b9c31b72a None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:39:25.214427 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33430a9d-975f-4c0d-9f96-8e3b9c31b72a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:39:25.246694 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.auth.core [None req-33430a9d-975f-4c0d-9f96-8e3b9c31b72a None None] MFA Rules not processed for user `0b4d91a8096f4f70983ebad7e097a78a`. Rule list: `[]` (Enabled: `True`). {{(pid=80119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:39:25.248078 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-33430a9d-975f-4c0d-9f96-8e3b9c31b72a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:39:25.328567 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1424/2843] 10.222.0.49 () {58 vars in 957 bytes} [Wed Jun 15 10:39:25 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:39:25.333112 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017ace4b-e272-4280-aa0b-7e11d92dbc9f service neutron] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:39:25.344176 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017ace4b-e272-4280-aa0b-7e11d92dbc9f service neutron] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:39:25.349307 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bcbcd8c-a4d0-4863-9049-62bc96fdc7bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:39:25.349685 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcbcd8c-a4d0-4863-9049-62bc96fdc7bf admin admin] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:39:25.349969 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcbcd8c-a4d0-4863-9049-62bc96fdc7bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:39:25.350248 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcbcd8c-a4d0-4863-9049-62bc96fdc7bf admin admin] PATH_INFO: `/v3/projects` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:39:25.350956 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bcbcd8c-a4d0-4863-9049-62bc96fdc7bf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:39:25.352936 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bcbcd8c-a4d0-4863-9049-62bc96fdc7bf admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:39:25.382455 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1420/2844] 10.222.0.49 () {60 vars in 1152 bytes} [Wed Jun 15 10:39:25 2022] POST /identity/v3/projects => generated 373 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 10:39:25.386278 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33430a9d-975f-4c0d-9f96-8e3b9c31b72a None None] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:39:25.404419 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33430a9d-975f-4c0d-9f96-8e3b9c31b72a None None] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:39:25.470788 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77555165-51cc-40e0-b886-948981b7b11a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:39:25.471525 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77555165-51cc-40e0-b886-948981b7b11a admin admin] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:39:25.471954 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77555165-51cc-40e0-b886-948981b7b11a admin admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:39:25.472251 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77555165-51cc-40e0-b886-948981b7b11a admin admin] PATH_INFO: `/v3/users` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:39:25.472951 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77555165-51cc-40e0-b886-948981b7b11a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:39:25.474894 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77555165-51cc-40e0-b886-948981b7b11a admin admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:39:25.502616 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1425/2845] 10.222.0.49 () {60 vars in 1143 bytes} [Wed Jun 15 10:39:25 2022] POST /identity/v3/users => generated 293 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 10:39:25.506882 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bcbcd8c-a4d0-4863-9049-62bc96fdc7bf admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:39:25.520023 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bcbcd8c-a4d0-4863-9049-62bc96fdc7bf admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:39:25.532141 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2c2581-06cc-44ad-95fb-bacbd9103166 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:39:25.532954 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2c2581-06cc-44ad-95fb-bacbd9103166 admin admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:39:25.533326 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2c2581-06cc-44ad-95fb-bacbd9103166 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:39:25.533708 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2c2581-06cc-44ad-95fb-bacbd9103166 admin admin] PATH_INFO: `/v3/roles` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:39:25.534452 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e2c2581-06cc-44ad-95fb-bacbd9103166 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:39:25.536142 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e2c2581-06cc-44ad-95fb-bacbd9103166 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:39:25.548123 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1421/2846] 10.222.0.49 () {58 vars in 1121 bytes} [Wed Jun 15 10:39:25 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 10:39:25.552262 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77555165-51cc-40e0-b886-948981b7b11a admin admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:39:25.563281 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77555165-51cc-40e0-b886-948981b7b11a admin admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:39:25.571412 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82435dd1-2d11-4db5-be84-cc54739b9d46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:39:25.572664 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82435dd1-2d11-4db5-be84-cc54739b9d46 admin admin] REQUEST_METHOD: `PUT` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:39:25.572955 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82435dd1-2d11-4db5-be84-cc54739b9d46 admin admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:39:25.573235 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82435dd1-2d11-4db5-be84-cc54739b9d46 admin admin] PATH_INFO: `/v3/projects/b5665e698ac049a0926aa5279d817c35/users/2f887b8c1031428ba655cad67f43f631/roles/8051174e7a7947d5b991ebd793c2e543` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:39:25.590270 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82435dd1-2d11-4db5-be84-cc54739b9d46 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b5665e698ac049a0926aa5279d817c35, user_id=2f887b8c1031428ba655cad67f43f631, role_id=8051174e7a7947d5b991ebd793c2e543)` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:39:25.593496 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82435dd1-2d11-4db5-be84-cc54739b9d46 admin admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:39:25.608372 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1426/2847] 10.222.0.49 () {60 vars in 1482 bytes} [Wed Jun 15 10:39:25 2022] PUT /identity/v3/projects/b5665e698ac049a0926aa5279d817c35/users/2f887b8c1031428ba655cad67f43f631/roles/8051174e7a7947d5b991ebd793c2e543 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:39:25.612859 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2c2581-06cc-44ad-95fb-bacbd9103166 admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:39:25.625901 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2c2581-06cc-44ad-95fb-bacbd9103166 admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:39:25.707571 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d824bd3-6776-4695-b8d4-82aead1858f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:39:25.708584 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d824bd3-6776-4695-b8d4-82aead1858f8 admin admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:39:25.708965 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d824bd3-6776-4695-b8d4-82aead1858f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:39:25.709336 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d824bd3-6776-4695-b8d4-82aead1858f8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:39:25.710216 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d824bd3-6776-4695-b8d4-82aead1858f8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:39:25.712112 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d824bd3-6776-4695-b8d4-82aead1858f8 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:39:25.723903 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1422/2848] 10.222.0.49 () {58 vars in 1121 bytes} [Wed Jun 15 10:39:25 2022] GET /identity/v3/roles => generated 1420 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 10:39:25.727964 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82435dd1-2d11-4db5-be84-cc54739b9d46 admin admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:39:25.742103 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82435dd1-2d11-4db5-be84-cc54739b9d46 admin admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:39:25.747051 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af7b983-e8e9-4ed5-905a-3fe57ff03e61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:39:25.748201 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af7b983-e8e9-4ed5-905a-3fe57ff03e61 admin admin] REQUEST_METHOD: `PUT` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:39:25.748556 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af7b983-e8e9-4ed5-905a-3fe57ff03e61 admin admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:39:25.748918 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af7b983-e8e9-4ed5-905a-3fe57ff03e61 admin admin] PATH_INFO: `/v3/projects/b5665e698ac049a0926aa5279d817c35/users/2f887b8c1031428ba655cad67f43f631/roles/8051174e7a7947d5b991ebd793c2e543` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:39:25.751064 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af7b983-e8e9-4ed5-905a-3fe57ff03e61 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b5665e698ac049a0926aa5279d817c35, user_id=2f887b8c1031428ba655cad67f43f631, role_id=8051174e7a7947d5b991ebd793c2e543)` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:39:25.753875 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af7b983-e8e9-4ed5-905a-3fe57ff03e61 admin admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:39:25.766369 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1427/2849] 10.222.0.49 () {60 vars in 1482 bytes} [Wed Jun 15 10:39:25 2022] PUT /identity/v3/projects/b5665e698ac049a0926aa5279d817c35/users/2f887b8c1031428ba655cad67f43f631/roles/8051174e7a7947d5b991ebd793c2e543 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 10:39:31.201956 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c921f68f-b9c5-4195-a605-8e71ef866588 None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:39:31.202500 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c921f68f-b9c5-4195-a605-8e71ef866588 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:39:31.202625 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c921f68f-b9c5-4195-a605-8e71ef866588 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:39:31.235114 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-c921f68f-b9c5-4195-a605-8e71ef866588 None None] MFA Rules not processed for user `2f887b8c1031428ba655cad67f43f631`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:39:31.236732 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-c921f68f-b9c5-4195-a605-8e71ef866588 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:39:31.312851 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:39:31.330961 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1423/2850] 10.222.0.49 () {58 vars in 957 bytes} [Wed Jun 15 10:39:31 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:39:31.334198 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af7b983-e8e9-4ed5-905a-3fe57ff03e61 admin admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:39:31.335986 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-1af7b983-e8e9-4ed5-905a-3fe57ff03e61 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:39:31.372081 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:39:31.437297 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af7b983-e8e9-4ed5-905a-3fe57ff03e61 admin admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:39:31.442825 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1311ded7-f44e-4d05-ba31-55fb57314a60 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ac54528370d4747b6e8777fba9d0c82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '559b3e713f5844b6ae37151d56dd419f', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:39:31.443888 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1311ded7-f44e-4d05-ba31-55fb57314a60 service nova] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:39:31.444200 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1311ded7-f44e-4d05-ba31-55fb57314a60 service nova] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:39:31.444486 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1311ded7-f44e-4d05-ba31-55fb57314a60 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:39:31.504667 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1311ded7-f44e-4d05-ba31-55fb57314a60 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:39:31.506476 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1311ded7-f44e-4d05-ba31-55fb57314a60 service nova] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:39:31.516479 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1428/2851] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Jun 15 10:39:31 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:39:32.774330 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02653ade-ccfa-4bd9-a31e-c67253d59b27 None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:39:32.774892 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02653ade-ccfa-4bd9-a31e-c67253d59b27 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:39:32.774892 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02653ade-ccfa-4bd9-a31e-c67253d59b27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:39:32.794246 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-02653ade-ccfa-4bd9-a31e-c67253d59b27 None None] MFA Rules not processed for user `0b4d91a8096f4f70983ebad7e097a78a`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:39:32.795515 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-02653ade-ccfa-4bd9-a31e-c67253d59b27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:39:32.802841 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1424/2852] 10.222.0.49 () {58 vars in 957 bytes} [Wed Jun 15 10:39:32 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:39:36.174402 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1311ded7-f44e-4d05-ba31-55fb57314a60 service nova] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:39:36.175991 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-1311ded7-f44e-4d05-ba31-55fb57314a60 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:39:36.237586 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1311ded7-f44e-4d05-ba31-55fb57314a60 service nova] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:39:36.244920 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b26f98-1d05-4770-9d08-c9e28b5b19a7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0cb44a17d205431793e9a8791e477a32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '559b3e713f5844b6ae37151d56dd419f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:39:36.250711 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b26f98-1d05-4770-9d08-c9e28b5b19a7 service neutron] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:39:36.250711 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b26f98-1d05-4770-9d08-c9e28b5b19a7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:39:36.250711 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b26f98-1d05-4770-9d08-c9e28b5b19a7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:39:36.252803 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-79b26f98-1d05-4770-9d08-c9e28b5b19a7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:39:36.328021 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b26f98-1d05-4770-9d08-c9e28b5b19a7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:39:36.329496 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b26f98-1d05-4770-9d08-c9e28b5b19a7 service neutron] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:39:36.339367 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1429/2853] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Jun 15 10:39:36 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:39:38.305000 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02653ade-ccfa-4bd9-a31e-c67253d59b27 None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:39:38.306440 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-02653ade-ccfa-4bd9-a31e-c67253d59b27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:39:38.322611 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02653ade-ccfa-4bd9-a31e-c67253d59b27 None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:39:38.328618 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b83e86a-8b57-4ea8-a1d8-ecc71c8783ed service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ac54528370d4747b6e8777fba9d0c82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '559b3e713f5844b6ae37151d56dd419f', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:39:38.329628 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b83e86a-8b57-4ea8-a1d8-ecc71c8783ed service nova] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:39:38.329628 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b83e86a-8b57-4ea8-a1d8-ecc71c8783ed service nova] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:39:38.329834 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b83e86a-8b57-4ea8-a1d8-ecc71c8783ed service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:39:38.331644 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-8b83e86a-8b57-4ea8-a1d8-ecc71c8783ed service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:39:38.344284 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b83e86a-8b57-4ea8-a1d8-ecc71c8783ed service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:39:38.348014 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b83e86a-8b57-4ea8-a1d8-ecc71c8783ed service nova] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:39:38.377730 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1425/2854] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Jun 15 10:39:38 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 73 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:39:41.877858 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b26f98-1d05-4770-9d08-c9e28b5b19a7 service neutron] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:39:41.879274 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-79b26f98-1d05-4770-9d08-c9e28b5b19a7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:39:41.889580 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=100 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:39:41.959493 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b26f98-1d05-4770-9d08-c9e28b5b19a7 service neutron] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:39:41.965526 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0110a90-6491-43a9-8e20-e3ada12cb8ce service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd03612a525644518517069b3fa43522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '559b3e713f5844b6ae37151d56dd419f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:39:41.966192 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0110a90-6491-43a9-8e20-e3ada12cb8ce service placement] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:39:41.966480 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0110a90-6491-43a9-8e20-e3ada12cb8ce service placement] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:39:41.966762 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0110a90-6491-43a9-8e20-e3ada12cb8ce service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:39:41.968663 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-c0110a90-6491-43a9-8e20-e3ada12cb8ce service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:39:41.980961 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0110a90-6491-43a9-8e20-e3ada12cb8ce service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:39:41.982564 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0110a90-6491-43a9-8e20-e3ada12cb8ce service placement] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:39:41.996686 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1430/2855] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Jun 15 10:39:41 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:39:48.325433 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600d620b-1ca0-41e3-8038-14caf75de8ea None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:39:48.325433 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600d620b-1ca0-41e3-8038-14caf75de8ea None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:39:48.325433 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600d620b-1ca0-41e3-8038-14caf75de8ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:39:48.343216 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-600d620b-1ca0-41e3-8038-14caf75de8ea None None] MFA Rules not processed for user `103e10244eca444b86b9c51ff314d445`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:39:48.344443 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-600d620b-1ca0-41e3-8038-14caf75de8ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:39:48.411007 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1426/2856] 10.222.0.49 () {58 vars in 1009 bytes} [Wed Jun 15 10:39:48 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:39:48.423593 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0110a90-6491-43a9-8e20-e3ada12cb8ce service placement] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:39:48.425026 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-c0110a90-6491-43a9-8e20-e3ada12cb8ce service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:39:48.496513 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0110a90-6491-43a9-8e20-e3ada12cb8ce service placement] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:39:48.503462 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1d4b1b-fb19-4509-bd40-bddd85963a7c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a69f99210444f9696e8b22615fb4a41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '559b3e713f5844b6ae37151d56dd419f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:39:48.504329 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1d4b1b-fb19-4509-bd40-bddd85963a7c service swift] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:39:48.504747 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1d4b1b-fb19-4509-bd40-bddd85963a7c service swift] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:39:48.505129 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1d4b1b-fb19-4509-bd40-bddd85963a7c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:39:48.515197 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e1d4b1b-fb19-4509-bd40-bddd85963a7c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:39:48.517377 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e1d4b1b-fb19-4509-bd40-bddd85963a7c service swift] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:39:48.527765 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1431/2857] 10.222.0.49 () {60 vars in 1384 bytes} [Wed Jun 15 10:39:48 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 15 10:39:58.404319 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=35 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:39:58.524010 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=61 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:40:04.382436 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f50cc1-693e-43fa-85ea-037064724508 None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:40:04.382857 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f50cc1-693e-43fa-85ea-037064724508 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:40:04.382988 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f50cc1-693e-43fa-85ea-037064724508 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:40:04.402332 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-43f50cc1-693e-43fa-85ea-037064724508 None None] MFA Rules not processed for user `103e10244eca444b86b9c51ff314d445`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:40:04.403676 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-43f50cc1-693e-43fa-85ea-037064724508 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:40:04.412272 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1427/2858] 10.222.0.49 () {58 vars in 967 bytes} [Wed Jun 15 10:40:04 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:40:04.425212 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1d4b1b-fb19-4509-bd40-bddd85963a7c service swift] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:40:04.436343 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1d4b1b-fb19-4509-bd40-bddd85963a7c service swift] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:40:04.441448 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ed838f-de4e-4517-a618-a61faf26bc46 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a69f99210444f9696e8b22615fb4a41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '559b3e713f5844b6ae37151d56dd419f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:40:04.442094 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ed838f-de4e-4517-a618-a61faf26bc46 service swift] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:40:04.442378 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ed838f-de4e-4517-a618-a61faf26bc46 service swift] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:40:04.442686 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ed838f-de4e-4517-a618-a61faf26bc46 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:40:04.450911 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ed838f-de4e-4517-a618-a61faf26bc46 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:40:04.452529 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ed838f-de4e-4517-a618-a61faf26bc46 service swift] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:40:04.460617 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1432/2859] 10.222.0.49 () {60 vars in 1384 bytes} [Wed Jun 15 10:40:04 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 15 10:40:06.381705 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f50cc1-693e-43fa-85ea-037064724508 None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:40:06.384094 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-43f50cc1-693e-43fa-85ea-037064724508 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:40:06.453126 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f50cc1-693e-43fa-85ea-037064724508 None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:40:06.457883 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209c32fa-99fb-456a-a207-33987a01bfe2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:40:06.458761 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209c32fa-99fb-456a-a207-33987a01bfe2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:40:06.459042 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209c32fa-99fb-456a-a207-33987a01bfe2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:40:06.459347 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209c32fa-99fb-456a-a207-33987a01bfe2 admin admin] PATH_INFO: `/v3/users/07276de63b65403585d0d7c576447f96` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:40:06.460569 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209c32fa-99fb-456a-a207-33987a01bfe2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=07276de63b65403585d0d7c576447f96)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:40:06.462507 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209c32fa-99fb-456a-a207-33987a01bfe2 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:40:06.514301 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-209c32fa-99fb-456a-a207-33987a01bfe2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '07276de63b65403585d0d7c576447f96'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:40:06.521111 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-209c32fa-99fb-456a-a207-33987a01bfe2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '07276de63b65403585d0d7c576447f96'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:40:06.521772 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-209c32fa-99fb-456a-a207-33987a01bfe2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '07276de63b65403585d0d7c576447f96'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:40:06.522656 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-209c32fa-99fb-456a-a207-33987a01bfe2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '07276de63b65403585d0d7c576447f96'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:40:06.529641 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-209c32fa-99fb-456a-a207-33987a01bfe2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '07276de63b65403585d0d7c576447f96'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:40:06.547319 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1428/2860] 10.222.0.49 () {58 vars in 1223 bytes} [Wed Jun 15 10:40:06 2022] DELETE /identity/v3/users/07276de63b65403585d0d7c576447f96 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:40:06.719118 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ed838f-de4e-4517-a618-a61faf26bc46 service swift] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:40:06.730971 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ed838f-de4e-4517-a618-a61faf26bc46 service swift] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:40:06.788790 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb7dcf7-872b-4de0-aa4b-f86e57bdf808 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:40:06.789756 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb7dcf7-872b-4de0-aa4b-f86e57bdf808 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:40:06.790105 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb7dcf7-872b-4de0-aa4b-f86e57bdf808 admin admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:40:06.790447 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb7dcf7-872b-4de0-aa4b-f86e57bdf808 admin admin] PATH_INFO: `/v3/projects/3efa3522561147ac8eff67781403423e` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:40:06.791738 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb7dcf7-872b-4de0-aa4b-f86e57bdf808 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3efa3522561147ac8eff67781403423e)` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:40:06.793707 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb7dcf7-872b-4de0-aa4b-f86e57bdf808 admin admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:40:06.839052 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-ceb7dcf7-872b-4de0-aa4b-f86e57bdf808 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3efa3522561147ac8eff67781403423e', 'request_id': 'req-ceb7dcf7-872b-4de0-aa4b-f86e57bdf808'} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:40:06.846331 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-ceb7dcf7-872b-4de0-aa4b-f86e57bdf808 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3efa3522561147ac8eff67781403423e', 'request_id': 'req-ceb7dcf7-872b-4de0-aa4b-f86e57bdf808'} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:40:06.851878 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-ceb7dcf7-872b-4de0-aa4b-f86e57bdf808 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3efa3522561147ac8eff67781403423e', 'request_id': 'req-ceb7dcf7-872b-4de0-aa4b-f86e57bdf808'} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:40:06.863235 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-ceb7dcf7-872b-4de0-aa4b-f86e57bdf808 admin admin] The token cache is being invalidate because project 3efa3522561147ac8eff67781403423e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80119) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 10:40:06.863573 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-ceb7dcf7-872b-4de0-aa4b-f86e57bdf808 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:40:06.864207 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-ceb7dcf7-872b-4de0-aa4b-f86e57bdf808 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:40:06.865738 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1433/2861] 10.222.0.49 () {58 vars in 1232 bytes} [Wed Jun 15 10:40:06 2022] DELETE /identity/v3/projects/3efa3522561147ac8eff67781403423e => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:40:16.543240 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=56,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:40:16.860211 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=57,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:40:19.557436 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27425ab9-b40c-4f4b-9984-86e92e7ce221 None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:40:19.557934 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27425ab9-b40c-4f4b-9984-86e92e7ce221 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:40:19.558041 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27425ab9-b40c-4f4b-9984-86e92e7ce221 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:40:19.574993 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-27425ab9-b40c-4f4b-9984-86e92e7ce221 None None] MFA Rules not processed for user `46fc1f62b79541e7a550731ee4728637`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:40:19.576238 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-27425ab9-b40c-4f4b-9984-86e92e7ce221 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:40:19.630755 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1429/2862] 10.222.0.49 () {58 vars in 957 bytes} [Wed Jun 15 10:40:19 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:40:21.857371 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb7dcf7-872b-4de0-aa4b-f86e57bdf808 admin admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:40:21.858804 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-ceb7dcf7-872b-4de0-aa4b-f86e57bdf808 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:40:21.915587 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb7dcf7-872b-4de0-aa4b-f86e57bdf808 admin admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:40:21.919225 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d961302-28f5-4af1-ba71-2add148d095e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0cb44a17d205431793e9a8791e477a32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '559b3e713f5844b6ae37151d56dd419f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:40:21.919889 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d961302-28f5-4af1-ba71-2add148d095e service neutron] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:40:21.920167 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d961302-28f5-4af1-ba71-2add148d095e service neutron] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:40:21.920448 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d961302-28f5-4af1-ba71-2add148d095e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:40:21.922763 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-5d961302-28f5-4af1-ba71-2add148d095e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:40:21.989912 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d961302-28f5-4af1-ba71-2add148d095e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:40:21.991306 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d961302-28f5-4af1-ba71-2add148d095e service neutron] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:40:22.001576 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1434/2863] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Jun 15 10:40:21 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:40:29.623517 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=25 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:40:31.996549 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=50 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:41:19.092045 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27425ab9-b40c-4f4b-9984-86e92e7ce221 None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:41:19.093627 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-27425ab9-b40c-4f4b-9984-86e92e7ce221 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:41:19.159295 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27425ab9-b40c-4f4b-9984-86e92e7ce221 None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:41:19.163933 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090ddcf8-7639-468f-add0-d3138af633e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:41:19.164804 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090ddcf8-7639-468f-add0-d3138af633e7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:41:19.165091 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090ddcf8-7639-468f-add0-d3138af633e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:41:19.165372 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090ddcf8-7639-468f-add0-d3138af633e7 admin admin] PATH_INFO: `/v3/users/46fc1f62b79541e7a550731ee4728637` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:41:19.166548 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-090ddcf8-7639-468f-add0-d3138af633e7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=46fc1f62b79541e7a550731ee4728637)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:41:19.168498 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-090ddcf8-7639-468f-add0-d3138af633e7 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:41:19.213157 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-090ddcf8-7639-468f-add0-d3138af633e7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '46fc1f62b79541e7a550731ee4728637'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:41:19.219683 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-090ddcf8-7639-468f-add0-d3138af633e7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '46fc1f62b79541e7a550731ee4728637'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:41:19.220346 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-090ddcf8-7639-468f-add0-d3138af633e7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '46fc1f62b79541e7a550731ee4728637'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:41:19.221433 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-090ddcf8-7639-468f-add0-d3138af633e7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '46fc1f62b79541e7a550731ee4728637'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:41:19.228343 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-090ddcf8-7639-468f-add0-d3138af633e7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '46fc1f62b79541e7a550731ee4728637'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:41:19.246186 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1430/2864] 10.222.0.49 () {58 vars in 1223 bytes} [Wed Jun 15 10:41:19 2022] DELETE /identity/v3/users/46fc1f62b79541e7a550731ee4728637 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:41:19.412037 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d961302-28f5-4af1-ba71-2add148d095e service neutron] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:41:19.422946 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d961302-28f5-4af1-ba71-2add148d095e service neutron] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:41:19.480502 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5b45e0-d0c7-412a-a4ad-5ade6766b0c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:41:19.481298 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5b45e0-d0c7-412a-a4ad-5ade6766b0c6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:41:19.481581 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5b45e0-d0c7-412a-a4ad-5ade6766b0c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:41:19.481858 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5b45e0-d0c7-412a-a4ad-5ade6766b0c6 admin admin] PATH_INFO: `/v3/projects/129b7faaaf264434baf88ffe0fe9bb25` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:41:19.483017 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5b45e0-d0c7-412a-a4ad-5ade6766b0c6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=129b7faaaf264434baf88ffe0fe9bb25)` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:41:19.485010 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5b45e0-d0c7-412a-a4ad-5ade6766b0c6 admin admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:41:19.530692 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-eb5b45e0-d0c7-412a-a4ad-5ade6766b0c6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '129b7faaaf264434baf88ffe0fe9bb25', 'request_id': 'req-eb5b45e0-d0c7-412a-a4ad-5ade6766b0c6'} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:41:19.538472 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-eb5b45e0-d0c7-412a-a4ad-5ade6766b0c6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '129b7faaaf264434baf88ffe0fe9bb25', 'request_id': 'req-eb5b45e0-d0c7-412a-a4ad-5ade6766b0c6'} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:41:19.544005 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-eb5b45e0-d0c7-412a-a4ad-5ade6766b0c6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '129b7faaaf264434baf88ffe0fe9bb25', 'request_id': 'req-eb5b45e0-d0c7-412a-a4ad-5ade6766b0c6'} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:41:19.554776 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-eb5b45e0-d0c7-412a-a4ad-5ade6766b0c6 admin admin] The token cache is being invalidate because project 129b7faaaf264434baf88ffe0fe9bb25 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80119) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 10:41:19.555204 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-eb5b45e0-d0c7-412a-a4ad-5ade6766b0c6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:41:19.555834 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-eb5b45e0-d0c7-412a-a4ad-5ade6766b0c6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:41:19.557351 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1435/2865] 10.222.0.49 () {58 vars in 1232 bytes} [Wed Jun 15 10:41:19 2022] DELETE /identity/v3/projects/129b7faaaf264434baf88ffe0fe9bb25 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:41:19.562697 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2d5c5c-74b8-455a-8c12-03a1b3be053b None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:41:19.562981 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2d5c5c-74b8-455a-8c12-03a1b3be053b None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:41:19.563284 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2d5c5c-74b8-455a-8c12-03a1b3be053b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:41:19.580783 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-1f2d5c5c-74b8-455a-8c12-03a1b3be053b None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:41:19.581986 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-1f2d5c5c-74b8-455a-8c12-03a1b3be053b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:41:19.638385 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1431/2866] 10.222.0.49 () {58 vars in 957 bytes} [Wed Jun 15 10:41:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:41:19.646550 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e368ad-0214-41eb-a6b3-b4b38dcd3a47 None None] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:41:19.646843 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e368ad-0214-41eb-a6b3-b4b38dcd3a47 None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:41:19.647155 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e368ad-0214-41eb-a6b3-b4b38dcd3a47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:41:19.665849 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.auth.core [None req-96e368ad-0214-41eb-a6b3-b4b38dcd3a47 None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:41:19.667062 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-96e368ad-0214-41eb-a6b3-b4b38dcd3a47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:41:19.675061 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1436/2867] 10.222.0.49 () {58 vars in 957 bytes} [Wed Jun 15 10:41:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:41:19.679041 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2d5c5c-74b8-455a-8c12-03a1b3be053b None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:41:19.688642 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2d5c5c-74b8-455a-8c12-03a1b3be053b None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:41:19.693856 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7fb1ab-7e15-4136-9b9e-2019ce6d931d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:41:19.694525 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7fb1ab-7e15-4136-9b9e-2019ce6d931d admin admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:41:19.694807 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7fb1ab-7e15-4136-9b9e-2019ce6d931d admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:41:19.695080 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7fb1ab-7e15-4136-9b9e-2019ce6d931d admin admin] PATH_INFO: `/v3/domains` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:41:19.695868 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d7fb1ab-7e15-4136-9b9e-2019ce6d931d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:41:19.696175 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d7fb1ab-7e15-4136-9b9e-2019ce6d931d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:41:19.697767 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d7fb1ab-7e15-4136-9b9e-2019ce6d931d admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:41:19.709876 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1432/2868] 10.222.0.49 () {58 vars in 1152 bytes} [Wed Jun 15 10:41:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:41:19.713853 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e368ad-0214-41eb-a6b3-b4b38dcd3a47 None None] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:41:19.723518 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e368ad-0214-41eb-a6b3-b4b38dcd3a47 None None] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:41:19.728172 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0a6f29-7d0a-44e8-b284-00fb0a89d27a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:41:19.728826 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0a6f29-7d0a-44e8-b284-00fb0a89d27a admin admin] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:41:19.729111 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0a6f29-7d0a-44e8-b284-00fb0a89d27a admin admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:41:19.729388 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0a6f29-7d0a-44e8-b284-00fb0a89d27a admin admin] PATH_INFO: `/v3/projects` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:41:19.730077 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0a6f29-7d0a-44e8-b284-00fb0a89d27a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:41:19.732069 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0a6f29-7d0a-44e8-b284-00fb0a89d27a admin admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:41:19.756918 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1437/2869] 10.222.0.49 () {60 vars in 1152 bytes} [Wed Jun 15 10:41:19 2022] POST /identity/v3/projects => generated 397 bytes in 43 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 10:41:19.761210 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7fb1ab-7e15-4136-9b9e-2019ce6d931d admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:41:19.772158 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7fb1ab-7e15-4136-9b9e-2019ce6d931d admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:41:19.775880 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:41:19.831128 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a393a8e9-d709-4a12-8a5c-025e637bbda5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:41:19.831790 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a393a8e9-d709-4a12-8a5c-025e637bbda5 admin admin] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:41:19.832074 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a393a8e9-d709-4a12-8a5c-025e637bbda5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:41:19.832345 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a393a8e9-d709-4a12-8a5c-025e637bbda5 admin admin] PATH_INFO: `/v3/users` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:41:19.833017 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a393a8e9-d709-4a12-8a5c-025e637bbda5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:41:19.834903 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a393a8e9-d709-4a12-8a5c-025e637bbda5 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:41:19.854328 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1433/2870] 10.222.0.49 () {60 vars in 1143 bytes} [Wed Jun 15 10:41:19 2022] POST /identity/v3/users => generated 299 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 10:41:19.858693 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0a6f29-7d0a-44e8-b284-00fb0a89d27a admin admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:41:19.869096 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0a6f29-7d0a-44e8-b284-00fb0a89d27a admin admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:41:19.873705 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f38eda-a8c8-4855-9f51-9e85d6e62c69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:41:19.874391 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f38eda-a8c8-4855-9f51-9e85d6e62c69 admin admin] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:41:19.874678 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f38eda-a8c8-4855-9f51-9e85d6e62c69 admin admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:41:19.874957 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f38eda-a8c8-4855-9f51-9e85d6e62c69 admin admin] PATH_INFO: `/v3/roles` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:41:19.875749 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f38eda-a8c8-4855-9f51-9e85d6e62c69 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:41:19.877874 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f38eda-a8c8-4855-9f51-9e85d6e62c69 admin admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:41:19.888093 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1438/2871] 10.222.0.49 () {58 vars in 1121 bytes} [Wed Jun 15 10:41:19 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 15 10:41:19.891835 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a393a8e9-d709-4a12-8a5c-025e637bbda5 admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:41:19.902691 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a393a8e9-d709-4a12-8a5c-025e637bbda5 admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:41:19.907240 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f64c635-6e72-418c-91cb-d072d681ecfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:41:19.907873 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f64c635-6e72-418c-91cb-d072d681ecfa admin admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:41:19.908159 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f64c635-6e72-418c-91cb-d072d681ecfa admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:41:19.908469 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f64c635-6e72-418c-91cb-d072d681ecfa admin admin] PATH_INFO: `/v3/roles` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:41:19.909186 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f64c635-6e72-418c-91cb-d072d681ecfa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:41:19.910742 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f64c635-6e72-418c-91cb-d072d681ecfa admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:41:19.920537 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1434/2872] 10.222.0.49 () {58 vars in 1121 bytes} [Wed Jun 15 10:41:19 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 10:41:19.926887 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f38eda-a8c8-4855-9f51-9e85d6e62c69 admin admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:41:19.937254 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f38eda-a8c8-4855-9f51-9e85d6e62c69 admin admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:41:19.942616 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396f9c38-4c06-4515-a3e2-42982e5c8f93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:41:19.943721 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396f9c38-4c06-4515-a3e2-42982e5c8f93 admin admin] REQUEST_METHOD: `PUT` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:41:19.944012 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396f9c38-4c06-4515-a3e2-42982e5c8f93 admin admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:41:19.944292 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396f9c38-4c06-4515-a3e2-42982e5c8f93 admin admin] PATH_INFO: `/v3/projects/9609ded88eff49a29e35928a9e47ee2f/users/db4b74bcdacd4b278bf5df7a620fc5db/roles/6876e57cc51c448097ab109d35b2a3da` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:41:19.959134 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396f9c38-4c06-4515-a3e2-42982e5c8f93 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9609ded88eff49a29e35928a9e47ee2f, user_id=db4b74bcdacd4b278bf5df7a620fc5db, role_id=6876e57cc51c448097ab109d35b2a3da)` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:41:19.961807 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396f9c38-4c06-4515-a3e2-42982e5c8f93 admin admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:41:19.973909 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1439/2873] 10.222.0.49 () {60 vars in 1482 bytes} [Wed Jun 15 10:41:19 2022] PUT /identity/v3/projects/9609ded88eff49a29e35928a9e47ee2f/users/db4b74bcdacd4b278bf5df7a620fc5db/roles/6876e57cc51c448097ab109d35b2a3da => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:41:19.984129 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac354fe-a5d3-4c2b-9dec-64c1b11d036c None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:41:19.984385 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac354fe-a5d3-4c2b-9dec-64c1b11d036c None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:41:19.984656 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac354fe-a5d3-4c2b-9dec-64c1b11d036c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:41:20.010600 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-9ac354fe-a5d3-4c2b-9dec-64c1b11d036c None None] MFA Rules not processed for user `db4b74bcdacd4b278bf5df7a620fc5db`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:41:20.011831 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-9ac354fe-a5d3-4c2b-9dec-64c1b11d036c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:41:20.074499 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1435/2874] 10.222.0.49 () {58 vars in 957 bytes} [Wed Jun 15 10:41:19 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:41:20.086663 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396f9c38-4c06-4515-a3e2-42982e5c8f93 admin admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:41:20.088386 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-396f9c38-4c06-4515-a3e2-42982e5c8f93 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:41:20.119394 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:41:20.147868 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396f9c38-4c06-4515-a3e2-42982e5c8f93 admin admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:41:20.151581 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b02666-5efb-4ea0-8bed-b67614a35f6e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20d1327553404b80aecee87c78fcb52b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '559b3e713f5844b6ae37151d56dd419f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:41:20.152219 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b02666-5efb-4ea0-8bed-b67614a35f6e service cinder] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:41:20.152497 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b02666-5efb-4ea0-8bed-b67614a35f6e service cinder] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:41:20.152768 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b02666-5efb-4ea0-8bed-b67614a35f6e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:41:20.160877 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b02666-5efb-4ea0-8bed-b67614a35f6e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:41:20.162297 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b02666-5efb-4ea0-8bed-b67614a35f6e service cinder] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:41:20.170928 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1440/2875] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Jun 15 10:41:20 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:41:27.148017 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac354fe-a5d3-4c2b-9dec-64c1b11d036c None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:41:27.208863 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac354fe-a5d3-4c2b-9dec-64c1b11d036c None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:41:27.213425 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab960e9d-b772-4fc3-a340-33338657ad73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:41:27.214295 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab960e9d-b772-4fc3-a340-33338657ad73 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:41:27.214578 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab960e9d-b772-4fc3-a340-33338657ad73 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:41:27.214850 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab960e9d-b772-4fc3-a340-33338657ad73 admin admin] PATH_INFO: `/v3/users/db4b74bcdacd4b278bf5df7a620fc5db` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:41:27.216134 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab960e9d-b772-4fc3-a340-33338657ad73 admin admin] RBAC: Authorizing `identity:delete_user(user_id=db4b74bcdacd4b278bf5df7a620fc5db)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:41:27.218070 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab960e9d-b772-4fc3-a340-33338657ad73 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:41:27.233769 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=96,keystone:INSERT=3,keystone:DELETE=1 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:41:27.263183 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-ab960e9d-b772-4fc3-a340-33338657ad73 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'db4b74bcdacd4b278bf5df7a620fc5db'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:41:27.270961 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-ab960e9d-b772-4fc3-a340-33338657ad73 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'db4b74bcdacd4b278bf5df7a620fc5db'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:41:27.271659 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-ab960e9d-b772-4fc3-a340-33338657ad73 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'db4b74bcdacd4b278bf5df7a620fc5db'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:41:27.272269 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-ab960e9d-b772-4fc3-a340-33338657ad73 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'db4b74bcdacd4b278bf5df7a620fc5db'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:41:27.280295 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-ab960e9d-b772-4fc3-a340-33338657ad73 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'db4b74bcdacd4b278bf5df7a620fc5db'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:41:27.297633 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1436/2876] 10.222.0.49 () {58 vars in 1223 bytes} [Wed Jun 15 10:41:27 2022] DELETE /identity/v3/users/db4b74bcdacd4b278bf5df7a620fc5db => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:41:27.302128 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d4cce4-f860-455c-8dad-8b69a02883d0 None None] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:41:27.302649 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d4cce4-f860-455c-8dad-8b69a02883d0 None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:41:27.302796 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d4cce4-f860-455c-8dad-8b69a02883d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:41:27.319153 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.auth.core [None req-72d4cce4-f860-455c-8dad-8b69a02883d0 None None] MFA Rules not processed for user `fcdb2d8e71bd4fe6b9b1431183facba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:41:27.320345 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-72d4cce4-f860-455c-8dad-8b69a02883d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:41:27.376938 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1441/2877] 10.222.0.49 () {58 vars in 957 bytes} [Wed Jun 15 10:41:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:41:27.386984 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab960e9d-b772-4fc3-a340-33338657ad73 admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:41:27.388445 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-ab960e9d-b772-4fc3-a340-33338657ad73 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:41:27.448362 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab960e9d-b772-4fc3-a340-33338657ad73 admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:41:27.452042 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1030eb-c7c3-41ce-9038-102dee87482e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0cb44a17d205431793e9a8791e477a32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '559b3e713f5844b6ae37151d56dd419f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:41:27.452685 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1030eb-c7c3-41ce-9038-102dee87482e service neutron] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:41:27.452967 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1030eb-c7c3-41ce-9038-102dee87482e service neutron] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:41:27.453239 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1030eb-c7c3-41ce-9038-102dee87482e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:41:27.462793 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c1030eb-c7c3-41ce-9038-102dee87482e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:41:27.464232 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c1030eb-c7c3-41ce-9038-102dee87482e service neutron] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:41:27.473942 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1437/2878] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Jun 15 10:41:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:41:27.739413 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d4cce4-f860-455c-8dad-8b69a02883d0 None None] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:41:27.740888 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-72d4cce4-f860-455c-8dad-8b69a02883d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:41:27.754463 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d4cce4-f860-455c-8dad-8b69a02883d0 None None] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:41:27.762704 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2b019d-3316-497b-9db2-e707902f1839 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:41:27.764036 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2b019d-3316-497b-9db2-e707902f1839 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:41:27.764036 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2b019d-3316-497b-9db2-e707902f1839 admin admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:41:27.764036 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2b019d-3316-497b-9db2-e707902f1839 admin admin] PATH_INFO: `/v3/projects/9609ded88eff49a29e35928a9e47ee2f` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:41:27.765290 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2b019d-3316-497b-9db2-e707902f1839 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9609ded88eff49a29e35928a9e47ee2f)` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:41:27.767271 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2b019d-3316-497b-9db2-e707902f1839 admin admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:41:27.835015 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-6c2b019d-3316-497b-9db2-e707902f1839 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9609ded88eff49a29e35928a9e47ee2f', 'request_id': 'req-6c2b019d-3316-497b-9db2-e707902f1839'} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:41:27.842264 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-6c2b019d-3316-497b-9db2-e707902f1839 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9609ded88eff49a29e35928a9e47ee2f', 'request_id': 'req-6c2b019d-3316-497b-9db2-e707902f1839'} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:41:27.847960 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-6c2b019d-3316-497b-9db2-e707902f1839 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9609ded88eff49a29e35928a9e47ee2f', 'request_id': 'req-6c2b019d-3316-497b-9db2-e707902f1839'} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:41:27.858887 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-6c2b019d-3316-497b-9db2-e707902f1839 admin admin] The token cache is being invalidate because project 9609ded88eff49a29e35928a9e47ee2f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80119) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 10:41:27.859208 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-6c2b019d-3316-497b-9db2-e707902f1839 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:41:27.859856 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-6c2b019d-3316-497b-9db2-e707902f1839 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:41:27.861433 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1442/2879] 10.222.0.49 () {58 vars in 1232 bytes} [Wed Jun 15 10:41:27 2022] DELETE /identity/v3/projects/9609ded88eff49a29e35928a9e47ee2f => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:41:37.469576 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:DELETE=9,keystone:SELECT=46,keystone:INSERT=1 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:41:37.855811 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=68,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:41:45.131956 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1030eb-c7c3-41ce-9038-102dee87482e service neutron] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:41:45.133412 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-7c1030eb-c7c3-41ce-9038-102dee87482e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:41:45.145171 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1030eb-c7c3-41ce-9038-102dee87482e service neutron] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:41:45.195320 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33b0f7b-7fbf-48fd-b730-95a9074e527b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0cb44a17d205431793e9a8791e477a32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '559b3e713f5844b6ae37151d56dd419f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:41:45.196018 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33b0f7b-7fbf-48fd-b730-95a9074e527b service neutron] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:41:45.196344 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33b0f7b-7fbf-48fd-b730-95a9074e527b service neutron] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:41:45.196624 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33b0f7b-7fbf-48fd-b730-95a9074e527b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:41:45.198427 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-e33b0f7b-7fbf-48fd-b730-95a9074e527b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:41:45.271654 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33b0f7b-7fbf-48fd-b730-95a9074e527b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:41:45.273085 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33b0f7b-7fbf-48fd-b730-95a9074e527b service neutron] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:41:45.284037 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1438/2880] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Jun 15 10:41:45 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:41:46.454635 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2b019d-3316-497b-9db2-e707902f1839 admin admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:41:46.456157 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-6c2b019d-3316-497b-9db2-e707902f1839 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:41:46.469085 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2b019d-3316-497b-9db2-e707902f1839 admin admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:41:46.472609 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3188f013-8799-4eec-bf54-f8ce0e8c99e3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0cb44a17d205431793e9a8791e477a32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '559b3e713f5844b6ae37151d56dd419f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:41:46.473821 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3188f013-8799-4eec-bf54-f8ce0e8c99e3 service neutron] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:41:46.474104 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3188f013-8799-4eec-bf54-f8ce0e8c99e3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:41:46.474378 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3188f013-8799-4eec-bf54-f8ce0e8c99e3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:41:46.476151 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-3188f013-8799-4eec-bf54-f8ce0e8c99e3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:41:46.489193 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3188f013-8799-4eec-bf54-f8ce0e8c99e3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:41:46.490605 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3188f013-8799-4eec-bf54-f8ce0e8c99e3 service neutron] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:41:46.501720 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1443/2881] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Jun 15 10:41:46 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:41:47.857084 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33b0f7b-7fbf-48fd-b730-95a9074e527b service neutron] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:41:47.858605 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-e33b0f7b-7fbf-48fd-b730-95a9074e527b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:41:47.873469 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33b0f7b-7fbf-48fd-b730-95a9074e527b service neutron] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:41:47.879008 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca66b02-7bbb-49d6-b6d0-1457e31d7cbd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ac54528370d4747b6e8777fba9d0c82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '559b3e713f5844b6ae37151d56dd419f', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:41:47.879698 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca66b02-7bbb-49d6-b6d0-1457e31d7cbd service nova] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:41:47.880583 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca66b02-7bbb-49d6-b6d0-1457e31d7cbd service nova] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:41:47.880864 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca66b02-7bbb-49d6-b6d0-1457e31d7cbd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:41:47.882671 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-7ca66b02-7bbb-49d6-b6d0-1457e31d7cbd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:41:47.896422 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca66b02-7bbb-49d6-b6d0-1457e31d7cbd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:41:47.898040 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca66b02-7bbb-49d6-b6d0-1457e31d7cbd service nova] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:41:47.909074 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1439/2882] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Jun 15 10:41:47 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:41:49.884082 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65335dd-7b88-45c9-8c05-6119e50e0e35 None None] REQUEST_METHOD: `POST` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:41:49.884561 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65335dd-7b88-45c9-8c05-6119e50e0e35 None None] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:41:49.884663 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65335dd-7b88-45c9-8c05-6119e50e0e35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:41:49.902447 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.auth.core [None req-a65335dd-7b88-45c9-8c05-6119e50e0e35 None None] MFA Rules not processed for user `61bc9ad2c9444fc297eb4b5902a67b98`. Rule list: `[]` (Enabled: `True`). {{(pid=80119) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:41:49.903780 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-a65335dd-7b88-45c9-8c05-6119e50e0e35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:41:49.959336 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1444/2883] 10.222.0.49 () {58 vars in 957 bytes} [Wed Jun 15 10:41:49 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:41:50.263599 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca66b02-7bbb-49d6-b6d0-1457e31d7cbd service nova] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:41:50.265033 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-7ca66b02-7bbb-49d6-b6d0-1457e31d7cbd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:41:50.276851 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca66b02-7bbb-49d6-b6d0-1457e31d7cbd service nova] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:41:50.280347 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7563cfc-cc85-40a7-8fdf-69b9321e2e54 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0cb44a17d205431793e9a8791e477a32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '559b3e713f5844b6ae37151d56dd419f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:41:50.280980 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7563cfc-cc85-40a7-8fdf-69b9321e2e54 service neutron] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:41:50.281265 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7563cfc-cc85-40a7-8fdf-69b9321e2e54 service neutron] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:41:50.281545 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7563cfc-cc85-40a7-8fdf-69b9321e2e54 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:41:50.283329 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-c7563cfc-cc85-40a7-8fdf-69b9321e2e54 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:41:50.293044 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7563cfc-cc85-40a7-8fdf-69b9321e2e54 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:41:50.294443 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7563cfc-cc85-40a7-8fdf-69b9321e2e54 service neutron] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:41:50.302889 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1440/2884] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Jun 15 10:41:50 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:41:50.740246 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65335dd-7b88-45c9-8c05-6119e50e0e35 None None] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:41:50.741745 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-a65335dd-7b88-45c9-8c05-6119e50e0e35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:41:50.812576 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65335dd-7b88-45c9-8c05-6119e50e0e35 None None] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:41:50.817460 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0872aa76-7951-4bd9-a0d1-5e4d317bed2a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd03612a525644518517069b3fa43522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '559b3e713f5844b6ae37151d56dd419f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:41:50.818130 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0872aa76-7951-4bd9-a0d1-5e4d317bed2a service placement] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:41:50.818415 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0872aa76-7951-4bd9-a0d1-5e4d317bed2a service placement] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:41:50.818692 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0872aa76-7951-4bd9-a0d1-5e4d317bed2a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:41:50.820476 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-0872aa76-7951-4bd9-a0d1-5e4d317bed2a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:41:50.831600 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0872aa76-7951-4bd9-a0d1-5e4d317bed2a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:41:50.833674 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0872aa76-7951-4bd9-a0d1-5e4d317bed2a service placement] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:41:50.843411 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1445/2885] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Jun 15 10:41:50 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:41:52.600583 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7563cfc-cc85-40a7-8fdf-69b9321e2e54 service neutron] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:41:52.612456 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7563cfc-cc85-40a7-8fdf-69b9321e2e54 service neutron] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:41:52.616613 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0793ad7b-1de6-43c0-b5e4-aa8877016d6d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0cb44a17d205431793e9a8791e477a32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '559b3e713f5844b6ae37151d56dd419f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:41:52.617279 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0793ad7b-1de6-43c0-b5e4-aa8877016d6d service neutron] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:41:52.617758 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0793ad7b-1de6-43c0-b5e4-aa8877016d6d service neutron] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:41:52.618041 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0793ad7b-1de6-43c0-b5e4-aa8877016d6d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:41:52.619988 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-0793ad7b-1de6-43c0-b5e4-aa8877016d6d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:41:52.631487 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0793ad7b-1de6-43c0-b5e4-aa8877016d6d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:41:52.632968 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0793ad7b-1de6-43c0-b5e4-aa8877016d6d service neutron] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:41:52.642068 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1441/2886] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Jun 15 10:41:52 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:42:00.839284 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=63 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:42:02.637684 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=72 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:42:04.113441 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0872aa76-7951-4bd9-a0d1-5e4d317bed2a service placement] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:42:04.115426 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-0872aa76-7951-4bd9-a0d1-5e4d317bed2a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:42:04.131345 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0872aa76-7951-4bd9-a0d1-5e4d317bed2a service placement] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:42:04.136792 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6437aac5-35a8-411f-bd99-1615293f9536 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ac54528370d4747b6e8777fba9d0c82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '559b3e713f5844b6ae37151d56dd419f', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:42:04.137435 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6437aac5-35a8-411f-bd99-1615293f9536 service nova] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:42:04.137719 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6437aac5-35a8-411f-bd99-1615293f9536 service nova] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:42:04.137996 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6437aac5-35a8-411f-bd99-1615293f9536 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:42:04.139757 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-6437aac5-35a8-411f-bd99-1615293f9536 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:42:04.152083 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6437aac5-35a8-411f-bd99-1615293f9536 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:42:04.153657 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6437aac5-35a8-411f-bd99-1615293f9536 service nova] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:42:04.164281 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1446/2887] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Jun 15 10:42:04 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:42:14.159772 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=6 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:42:43.983780 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0793ad7b-1de6-43c0-b5e4-aa8877016d6d service neutron] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:42:43.993091 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0793ad7b-1de6-43c0-b5e4-aa8877016d6d service neutron] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:42:43.996772 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5503a255-4d9f-4b85-9552-8868788b53f4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0cb44a17d205431793e9a8791e477a32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '559b3e713f5844b6ae37151d56dd419f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:42:43.997415 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5503a255-4d9f-4b85-9552-8868788b53f4 service neutron] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:42:43.997710 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5503a255-4d9f-4b85-9552-8868788b53f4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:42:43.997992 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5503a255-4d9f-4b85-9552-8868788b53f4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:42:43.999803 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-5503a255-4d9f-4b85-9552-8868788b53f4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:42:44.009798 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5503a255-4d9f-4b85-9552-8868788b53f4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:42:44.011275 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5503a255-4d9f-4b85-9552-8868788b53f4 service neutron] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:42:44.019867 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1442/2888] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Jun 15 10:42:43 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:42:45.101307 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6437aac5-35a8-411f-bd99-1615293f9536 service nova] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:42:45.110726 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6437aac5-35a8-411f-bd99-1615293f9536 service nova] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:42:45.114321 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841854c2-d016-42e8-8911-34537648613f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0cb44a17d205431793e9a8791e477a32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '559b3e713f5844b6ae37151d56dd419f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:42:45.114961 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841854c2-d016-42e8-8911-34537648613f service neutron] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:42:45.115291 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841854c2-d016-42e8-8911-34537648613f service neutron] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:42:45.115571 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841854c2-d016-42e8-8911-34537648613f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:42:45.117381 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-841854c2-d016-42e8-8911-34537648613f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:42:45.131525 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-841854c2-d016-42e8-8911-34537648613f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:42:45.133493 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-841854c2-d016-42e8-8911-34537648613f service neutron] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:42:45.145044 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1447/2889] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Jun 15 10:42:45 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:42:54.015884 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=6 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:42:55.140420 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=6 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:43:45.066203 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4347b0e-a043-4833-9903-7a413c4592b9 None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:43:45.066741 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4347b0e-a043-4833-9903-7a413c4592b9 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:43:45.066850 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4347b0e-a043-4833-9903-7a413c4592b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:43:45.085019 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-c4347b0e-a043-4833-9903-7a413c4592b9 None None] MFA Rules not processed for user `61bc9ad2c9444fc297eb4b5902a67b98`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:43:45.086251 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-c4347b0e-a043-4833-9903-7a413c4592b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:43:45.093901 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1443/2890] 10.222.0.49 () {58 vars in 957 bytes} [Wed Jun 15 10:43:45 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:43:53.834234 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841854c2-d016-42e8-8911-34537648613f service neutron] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:43:53.875296 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841854c2-d016-42e8-8911-34537648613f service neutron] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:43:53.880804 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a20ba19-db16-4370-826d-2dddedc2387e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ac54528370d4747b6e8777fba9d0c82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '559b3e713f5844b6ae37151d56dd419f', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:43:53.881440 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a20ba19-db16-4370-826d-2dddedc2387e service nova] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:43:53.881723 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a20ba19-db16-4370-826d-2dddedc2387e service nova] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:43:53.881999 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a20ba19-db16-4370-826d-2dddedc2387e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:43:53.883770 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-2a20ba19-db16-4370-826d-2dddedc2387e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:43:53.903136 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a20ba19-db16-4370-826d-2dddedc2387e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:43:53.904716 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a20ba19-db16-4370-826d-2dddedc2387e service nova] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:43:53.913581 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1448/2891] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Jun 15 10:43:53 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 80 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:43:55.079325 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=3 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:44:03.909430 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=24 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:44:21.412706 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4347b0e-a043-4833-9903-7a413c4592b9 None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:44:21.445729 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4347b0e-a043-4833-9903-7a413c4592b9 None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:44:21.449364 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c9713f-a269-4233-8b1a-19d596c02b8f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0cb44a17d205431793e9a8791e477a32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '559b3e713f5844b6ae37151d56dd419f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:44:21.450002 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c9713f-a269-4233-8b1a-19d596c02b8f service neutron] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:44:21.450281 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c9713f-a269-4233-8b1a-19d596c02b8f service neutron] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:44:21.450573 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c9713f-a269-4233-8b1a-19d596c02b8f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:44:21.452308 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-99c9713f-a269-4233-8b1a-19d596c02b8f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:44:21.538033 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c9713f-a269-4233-8b1a-19d596c02b8f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:44:21.539499 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c9713f-a269-4233-8b1a-19d596c02b8f service neutron] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:44:21.557654 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1444/2892] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Jun 15 10:44:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:44:24.546667 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a20ba19-db16-4370-826d-2dddedc2387e service nova] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:44:24.558092 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a20ba19-db16-4370-826d-2dddedc2387e service nova] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:44:24.563012 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b8fb2c-16d4-4d10-93e8-9acb313e35a0 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd03612a525644518517069b3fa43522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '559b3e713f5844b6ae37151d56dd419f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:44:24.563688 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b8fb2c-16d4-4d10-93e8-9acb313e35a0 service placement] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:44:24.563971 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b8fb2c-16d4-4d10-93e8-9acb313e35a0 service placement] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:44:24.564243 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b8fb2c-16d4-4d10-93e8-9acb313e35a0 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:44:24.565987 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-a8b8fb2c-16d4-4d10-93e8-9acb313e35a0 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:44:24.580205 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b8fb2c-16d4-4d10-93e8-9acb313e35a0 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:44:24.581773 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b8fb2c-16d4-4d10-93e8-9acb313e35a0 service placement] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:44:24.592745 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1449/2893] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Jun 15 10:44:24 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:44:27.651857 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c9713f-a269-4233-8b1a-19d596c02b8f service neutron] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:44:27.653568 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-99c9713f-a269-4233-8b1a-19d596c02b8f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:44:27.666787 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c9713f-a269-4233-8b1a-19d596c02b8f service neutron] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:44:27.672404 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1a313f-4f92-45b7-875b-9054d7432f3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:44:27.673394 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1a313f-4f92-45b7-875b-9054d7432f3f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:44:27.673741 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1a313f-4f92-45b7-875b-9054d7432f3f admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:44:27.674040 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1a313f-4f92-45b7-875b-9054d7432f3f admin admin] PATH_INFO: `/v3/users/0b4d91a8096f4f70983ebad7e097a78a` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:44:27.675367 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b1a313f-4f92-45b7-875b-9054d7432f3f admin admin] RBAC: Authorizing `identity:delete_user(user_id=0b4d91a8096f4f70983ebad7e097a78a)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:44:27.677311 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b1a313f-4f92-45b7-875b-9054d7432f3f admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:44:27.721721 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-4b1a313f-4f92-45b7-875b-9054d7432f3f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0b4d91a8096f4f70983ebad7e097a78a'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:44:27.728575 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-4b1a313f-4f92-45b7-875b-9054d7432f3f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0b4d91a8096f4f70983ebad7e097a78a'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:44:27.729237 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-4b1a313f-4f92-45b7-875b-9054d7432f3f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0b4d91a8096f4f70983ebad7e097a78a'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:44:27.730364 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-4b1a313f-4f92-45b7-875b-9054d7432f3f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0b4d91a8096f4f70983ebad7e097a78a'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:44:27.737209 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-4b1a313f-4f92-45b7-875b-9054d7432f3f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0b4d91a8096f4f70983ebad7e097a78a'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:44:27.754678 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1445/2894] 10.222.0.49 () {58 vars in 1223 bytes} [Wed Jun 15 10:44:27 2022] DELETE /identity/v3/users/0b4d91a8096f4f70983ebad7e097a78a => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:44:27.931465 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b8fb2c-16d4-4d10-93e8-9acb313e35a0 service placement] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:44:27.945243 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b8fb2c-16d4-4d10-93e8-9acb313e35a0 service placement] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:44:28.005609 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f559bc-71c6-48f5-aa4e-02a98c931011 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:44:28.006470 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f559bc-71c6-48f5-aa4e-02a98c931011 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:44:28.006754 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f559bc-71c6-48f5-aa4e-02a98c931011 admin admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:44:28.007031 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f559bc-71c6-48f5-aa4e-02a98c931011 admin admin] PATH_INFO: `/v3/projects/fe56b435fdbf49629bfc5e06c24e78b9` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:44:28.008220 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f559bc-71c6-48f5-aa4e-02a98c931011 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fe56b435fdbf49629bfc5e06c24e78b9)` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:44:28.010223 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f559bc-71c6-48f5-aa4e-02a98c931011 admin admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:44:28.056105 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-d0f559bc-71c6-48f5-aa4e-02a98c931011 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fe56b435fdbf49629bfc5e06c24e78b9', 'request_id': 'req-d0f559bc-71c6-48f5-aa4e-02a98c931011'} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:44:28.063835 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-d0f559bc-71c6-48f5-aa4e-02a98c931011 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fe56b435fdbf49629bfc5e06c24e78b9', 'request_id': 'req-d0f559bc-71c6-48f5-aa4e-02a98c931011'} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:44:28.069270 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-d0f559bc-71c6-48f5-aa4e-02a98c931011 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fe56b435fdbf49629bfc5e06c24e78b9', 'request_id': 'req-d0f559bc-71c6-48f5-aa4e-02a98c931011'} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:44:28.080075 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-d0f559bc-71c6-48f5-aa4e-02a98c931011 admin admin] The token cache is being invalidate because project fe56b435fdbf49629bfc5e06c24e78b9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80119) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 10:44:28.080363 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-d0f559bc-71c6-48f5-aa4e-02a98c931011 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:44:28.080987 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-d0f559bc-71c6-48f5-aa4e-02a98c931011 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:44:28.082522 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1450/2895] 10.222.0.49 () {58 vars in 1232 bytes} [Wed Jun 15 10:44:27 2022] DELETE /identity/v3/projects/fe56b435fdbf49629bfc5e06c24e78b9 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:44:28.086679 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1a313f-4f92-45b7-875b-9054d7432f3f admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:44:28.088119 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-4b1a313f-4f92-45b7-875b-9054d7432f3f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:44:28.101517 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1a313f-4f92-45b7-875b-9054d7432f3f admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:44:28.114129 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:44:28.161123 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79149ab7-758a-4c4e-b2c1-018c0c93ed8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:44:28.162001 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79149ab7-758a-4c4e-b2c1-018c0c93ed8a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:44:28.162293 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79149ab7-758a-4c4e-b2c1-018c0c93ed8a admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:44:28.162576 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79149ab7-758a-4c4e-b2c1-018c0c93ed8a admin admin] PATH_INFO: `/v3/users/2f887b8c1031428ba655cad67f43f631` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:44:28.163781 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79149ab7-758a-4c4e-b2c1-018c0c93ed8a admin admin] RBAC: Authorizing `identity:delete_user(user_id=2f887b8c1031428ba655cad67f43f631)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:44:28.165698 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79149ab7-758a-4c4e-b2c1-018c0c93ed8a admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:44:28.210134 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-79149ab7-758a-4c4e-b2c1-018c0c93ed8a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2f887b8c1031428ba655cad67f43f631'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:44:28.217409 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-79149ab7-758a-4c4e-b2c1-018c0c93ed8a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2f887b8c1031428ba655cad67f43f631'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:44:28.218059 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-79149ab7-758a-4c4e-b2c1-018c0c93ed8a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2f887b8c1031428ba655cad67f43f631'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:44:28.218664 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-79149ab7-758a-4c4e-b2c1-018c0c93ed8a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2f887b8c1031428ba655cad67f43f631'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:44:28.225514 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-79149ab7-758a-4c4e-b2c1-018c0c93ed8a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2f887b8c1031428ba655cad67f43f631'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:44:28.242665 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1446/2896] 10.222.0.49 () {58 vars in 1223 bytes} [Wed Jun 15 10:44:28 2022] DELETE /identity/v3/users/2f887b8c1031428ba655cad67f43f631 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:44:28.407921 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f559bc-71c6-48f5-aa4e-02a98c931011 admin admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:44:28.409377 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-d0f559bc-71c6-48f5-aa4e-02a98c931011 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:44:28.425238 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f559bc-71c6-48f5-aa4e-02a98c931011 admin admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:44:28.483683 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37ad51a-e828-44c6-b945-bf6596541834 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:44:28.484562 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37ad51a-e828-44c6-b945-bf6596541834 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:44:28.484848 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37ad51a-e828-44c6-b945-bf6596541834 admin admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:44:28.485127 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37ad51a-e828-44c6-b945-bf6596541834 admin admin] PATH_INFO: `/v3/projects/b5665e698ac049a0926aa5279d817c35` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:44:28.486304 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c37ad51a-e828-44c6-b945-bf6596541834 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b5665e698ac049a0926aa5279d817c35)` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:44:28.488270 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c37ad51a-e828-44c6-b945-bf6596541834 admin admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:44:28.508719 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:44:28.535536 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-c37ad51a-e828-44c6-b945-bf6596541834 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b5665e698ac049a0926aa5279d817c35', 'request_id': 'req-c37ad51a-e828-44c6-b945-bf6596541834'} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:44:28.542357 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-c37ad51a-e828-44c6-b945-bf6596541834 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b5665e698ac049a0926aa5279d817c35', 'request_id': 'req-c37ad51a-e828-44c6-b945-bf6596541834'} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:44:28.547866 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-c37ad51a-e828-44c6-b945-bf6596541834 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b5665e698ac049a0926aa5279d817c35', 'request_id': 'req-c37ad51a-e828-44c6-b945-bf6596541834'} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:44:28.559078 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-c37ad51a-e828-44c6-b945-bf6596541834 admin admin] The token cache is being invalidate because project b5665e698ac049a0926aa5279d817c35 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80119) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 10:44:28.559388 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-c37ad51a-e828-44c6-b945-bf6596541834 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:44:28.560011 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-c37ad51a-e828-44c6-b945-bf6596541834 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:44:28.561532 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1451/2897] 10.222.0.49 () {58 vars in 1232 bytes} [Wed Jun 15 10:44:28 2022] DELETE /identity/v3/projects/b5665e698ac049a0926aa5279d817c35 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:44:38.239160 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=45,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:44:38.555604 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=16,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:45:28.833085 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79149ab7-758a-4c4e-b2c1-018c0c93ed8a admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:45:28.834521 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-79149ab7-758a-4c4e-b2c1-018c0c93ed8a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:45:28.922929 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79149ab7-758a-4c4e-b2c1-018c0c93ed8a admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:45:28.979444 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd52dcf0-ac9c-44a7-a6d5-2a7012f39dd4 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd03612a525644518517069b3fa43522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '559b3e713f5844b6ae37151d56dd419f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:45:28.980027 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd52dcf0-ac9c-44a7-a6d5-2a7012f39dd4 service placement] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:45:28.980313 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd52dcf0-ac9c-44a7-a6d5-2a7012f39dd4 service placement] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:45:28.980591 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd52dcf0-ac9c-44a7-a6d5-2a7012f39dd4 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:45:28.982368 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-cd52dcf0-ac9c-44a7-a6d5-2a7012f39dd4 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:45:28.994013 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd52dcf0-ac9c-44a7-a6d5-2a7012f39dd4 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:45:28.995636 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd52dcf0-ac9c-44a7-a6d5-2a7012f39dd4 service placement] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:45:29.006663 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1447/2898] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Jun 15 10:45:28 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 174 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:45:39.001982 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=67 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:45:46.133862 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37ad51a-e828-44c6-b945-bf6596541834 admin admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:45:46.135347 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-c37ad51a-e828-44c6-b945-bf6596541834 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:45:46.204686 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37ad51a-e828-44c6-b945-bf6596541834 admin admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:45:46.208395 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ac2ff2-1549-4246-bee9-854aa1b90add service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0cb44a17d205431793e9a8791e477a32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '559b3e713f5844b6ae37151d56dd419f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:45:46.209044 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ac2ff2-1549-4246-bee9-854aa1b90add service neutron] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:45:46.209330 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ac2ff2-1549-4246-bee9-854aa1b90add service neutron] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:45:46.209606 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ac2ff2-1549-4246-bee9-854aa1b90add service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:45:46.211420 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-20ac2ff2-1549-4246-bee9-854aa1b90add service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:45:46.272619 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ac2ff2-1549-4246-bee9-854aa1b90add service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:45:46.274055 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ac2ff2-1549-4246-bee9-854aa1b90add service neutron] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:45:46.283584 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1452/2899] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Jun 15 10:45:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:45:52.286759 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd52dcf0-ac9c-44a7-a6d5-2a7012f39dd4 service placement] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:45:52.288804 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-cd52dcf0-ac9c-44a7-a6d5-2a7012f39dd4 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:45:52.302102 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd52dcf0-ac9c-44a7-a6d5-2a7012f39dd4 service placement] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:45:52.307724 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8e04e0-0640-42a1-b0e1-b8b2218283d1 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd03612a525644518517069b3fa43522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '559b3e713f5844b6ae37151d56dd419f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:45:52.308866 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8e04e0-0640-42a1-b0e1-b8b2218283d1 service placement] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:45:52.309335 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8e04e0-0640-42a1-b0e1-b8b2218283d1 service placement] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:45:52.309775 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8e04e0-0640-42a1-b0e1-b8b2218283d1 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:45:52.312271 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-bd8e04e0-0640-42a1-b0e1-b8b2218283d1 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:45:52.330045 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8e04e0-0640-42a1-b0e1-b8b2218283d1 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:45:52.332338 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8e04e0-0640-42a1-b0e1-b8b2218283d1 service placement] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:45:52.345008 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1448/2900] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Jun 15 10:45:52 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:45:52.350658 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ac2ff2-1549-4246-bee9-854aa1b90add service neutron] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:45:52.352938 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-20ac2ff2-1549-4246-bee9-854aa1b90add service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:45:52.371450 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ac2ff2-1549-4246-bee9-854aa1b90add service neutron] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:45:52.376355 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6deb9643-6676-4b89-84f5-306071bf19ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:45:52.377843 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6deb9643-6676-4b89-84f5-306071bf19ae admin admin] REQUEST_METHOD: `DELETE` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:45:52.378186 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6deb9643-6676-4b89-84f5-306071bf19ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:45:52.378525 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6deb9643-6676-4b89-84f5-306071bf19ae admin admin] PATH_INFO: `/v3/users/61bc9ad2c9444fc297eb4b5902a67b98` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:45:52.379964 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6deb9643-6676-4b89-84f5-306071bf19ae admin admin] RBAC: Authorizing `identity:delete_user(user_id=61bc9ad2c9444fc297eb4b5902a67b98)` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:45:52.382405 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6deb9643-6676-4b89-84f5-306071bf19ae admin admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:45:52.435223 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-6deb9643-6676-4b89-84f5-306071bf19ae admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '61bc9ad2c9444fc297eb4b5902a67b98'} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:45:52.442041 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-6deb9643-6676-4b89-84f5-306071bf19ae admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '61bc9ad2c9444fc297eb4b5902a67b98'} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:45:52.442715 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-6deb9643-6676-4b89-84f5-306071bf19ae admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '61bc9ad2c9444fc297eb4b5902a67b98'} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:45:52.443352 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-6deb9643-6676-4b89-84f5-306071bf19ae admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '61bc9ad2c9444fc297eb4b5902a67b98'} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:45:52.450651 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-6deb9643-6676-4b89-84f5-306071bf19ae admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '61bc9ad2c9444fc297eb4b5902a67b98'} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:45:52.467584 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1453/2901] 10.222.0.49 () {58 vars in 1223 bytes} [Wed Jun 15 10:45:52 2022] DELETE /identity/v3/users/61bc9ad2c9444fc297eb4b5902a67b98 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:45:52.639542 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8e04e0-0640-42a1-b0e1-b8b2218283d1 service placement] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:45:52.650338 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8e04e0-0640-42a1-b0e1-b8b2218283d1 service placement] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:45:52.706947 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d34f6f-9e3a-46a3-86b0-2b44096062fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:45:52.707829 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d34f6f-9e3a-46a3-86b0-2b44096062fb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:45:52.708112 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d34f6f-9e3a-46a3-86b0-2b44096062fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:45:52.708379 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d34f6f-9e3a-46a3-86b0-2b44096062fb admin admin] PATH_INFO: `/v3/projects/f096e399deac4d029ff63e3cb7f1ce9f` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:45:52.709592 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d34f6f-9e3a-46a3-86b0-2b44096062fb admin admin] RBAC: Authorizing `identity:delete_project(project_id=f096e399deac4d029ff63e3cb7f1ce9f)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:45:52.711538 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d34f6f-9e3a-46a3-86b0-2b44096062fb admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:45:52.755065 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-74d34f6f-9e3a-46a3-86b0-2b44096062fb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f096e399deac4d029ff63e3cb7f1ce9f', 'request_id': 'req-74d34f6f-9e3a-46a3-86b0-2b44096062fb'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:45:52.762793 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-74d34f6f-9e3a-46a3-86b0-2b44096062fb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f096e399deac4d029ff63e3cb7f1ce9f', 'request_id': 'req-74d34f6f-9e3a-46a3-86b0-2b44096062fb'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:45:52.774786 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-74d34f6f-9e3a-46a3-86b0-2b44096062fb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f096e399deac4d029ff63e3cb7f1ce9f', 'request_id': 'req-74d34f6f-9e3a-46a3-86b0-2b44096062fb'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:45:52.787908 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-74d34f6f-9e3a-46a3-86b0-2b44096062fb admin admin] The token cache is being invalidate because project f096e399deac4d029ff63e3cb7f1ce9f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80120) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 10:45:52.788640 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-74d34f6f-9e3a-46a3-86b0-2b44096062fb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:45:52.789305 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-74d34f6f-9e3a-46a3-86b0-2b44096062fb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:45:52.790853 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1449/2902] 10.222.0.49 () {58 vars in 1232 bytes} [Wed Jun 15 10:45:52 2022] DELETE /identity/v3/projects/f096e399deac4d029ff63e3cb7f1ce9f => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:45:52.795014 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6deb9643-6676-4b89-84f5-306071bf19ae admin admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:45:52.796251 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-6deb9643-6676-4b89-84f5-306071bf19ae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:45:52.809830 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6deb9643-6676-4b89-84f5-306071bf19ae admin admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:45:52.817756 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:45:52.867526 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71be0392-21e5-4d79-9e62-1ae632604ae2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:45:52.868395 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71be0392-21e5-4d79-9e62-1ae632604ae2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:45:52.868679 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71be0392-21e5-4d79-9e62-1ae632604ae2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:45:52.868959 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71be0392-21e5-4d79-9e62-1ae632604ae2 admin admin] PATH_INFO: `/v3/users/43929157365d4fad979f66e227572393` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:45:52.870156 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71be0392-21e5-4d79-9e62-1ae632604ae2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=43929157365d4fad979f66e227572393)` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:45:52.872129 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71be0392-21e5-4d79-9e62-1ae632604ae2 admin admin] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:45:52.918100 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-71be0392-21e5-4d79-9e62-1ae632604ae2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '43929157365d4fad979f66e227572393'} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:45:52.924764 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-71be0392-21e5-4d79-9e62-1ae632604ae2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '43929157365d4fad979f66e227572393'} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:45:52.925451 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-71be0392-21e5-4d79-9e62-1ae632604ae2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '43929157365d4fad979f66e227572393'} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:45:52.926055 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-71be0392-21e5-4d79-9e62-1ae632604ae2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '43929157365d4fad979f66e227572393'} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:45:52.933106 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.notifications [None req-71be0392-21e5-4d79-9e62-1ae632604ae2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '43929157365d4fad979f66e227572393'} {{(pid=80119) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:45:52.949885 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1454/2903] 10.222.0.49 () {58 vars in 1223 bytes} [Wed Jun 15 10:45:52 2022] DELETE /identity/v3/users/43929157365d4fad979f66e227572393 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:45:53.119235 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d34f6f-9e3a-46a3-86b0-2b44096062fb admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:45:53.121645 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-74d34f6f-9e3a-46a3-86b0-2b44096062fb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:45:53.136772 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d34f6f-9e3a-46a3-86b0-2b44096062fb admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:45:53.193191 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd13c10-98ee-48b2-b0ec-ae1b869c4606 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcdb2d8e71bd4fe6b9b1431183facba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8da8e1de64f469c9cc5f9a5f416060d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:45:53.193982 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd13c10-98ee-48b2-b0ec-ae1b869c4606 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:45:53.194265 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd13c10-98ee-48b2-b0ec-ae1b869c4606 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:45:53.194538 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd13c10-98ee-48b2-b0ec-ae1b869c4606 admin admin] PATH_INFO: `/v3/projects/ad67bfe861ba491c89827c72982ea8c2` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:45:53.195739 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd13c10-98ee-48b2-b0ec-ae1b869c4606 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ad67bfe861ba491c89827c72982ea8c2)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:45:53.197617 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd13c10-98ee-48b2-b0ec-ae1b869c4606 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:45:53.219004 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:45:53.243490 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-1cd13c10-98ee-48b2-b0ec-ae1b869c4606 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ad67bfe861ba491c89827c72982ea8c2', 'request_id': 'req-1cd13c10-98ee-48b2-b0ec-ae1b869c4606'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:45:53.250260 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-1cd13c10-98ee-48b2-b0ec-ae1b869c4606 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ad67bfe861ba491c89827c72982ea8c2', 'request_id': 'req-1cd13c10-98ee-48b2-b0ec-ae1b869c4606'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:45:53.256369 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-1cd13c10-98ee-48b2-b0ec-ae1b869c4606 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ad67bfe861ba491c89827c72982ea8c2', 'request_id': 'req-1cd13c10-98ee-48b2-b0ec-ae1b869c4606'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:45:53.267292 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-1cd13c10-98ee-48b2-b0ec-ae1b869c4606 admin admin] The token cache is being invalidate because project ad67bfe861ba491c89827c72982ea8c2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80120) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 10:45:53.267580 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-1cd13c10-98ee-48b2-b0ec-ae1b869c4606 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:45:53.268816 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-1cd13c10-98ee-48b2-b0ec-ae1b869c4606 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:45:53.270320 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1450/2904] 10.222.0.49 () {58 vars in 1232 bytes} [Wed Jun 15 10:45:53 2022] DELETE /identity/v3/projects/ad67bfe861ba491c89827c72982ea8c2 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:46:02.946331 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=48,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:46:03.264103 dv-cinder-843493-2-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=16,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:50:34.767556 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71be0392-21e5-4d79-9e62-1ae632604ae2 admin admin] Authenticating user token {{(pid=80119) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:50:34.768977 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-71be0392-21e5-4d79-9e62-1ae632604ae2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:50:34.837458 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71be0392-21e5-4d79-9e62-1ae632604ae2 admin admin] Validating token access rules against request {{(pid=80119) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:50:34.895659 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30cba9fc-2ec7-4ca3-97d5-5050f19a7781 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd03612a525644518517069b3fa43522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '559b3e713f5844b6ae37151d56dd419f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80119) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:50:34.896337 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cba9fc-2ec7-4ca3-97d5-5050f19a7781 service placement] REQUEST_METHOD: `GET` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:50:34.896621 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cba9fc-2ec7-4ca3-97d5-5050f19a7781 service placement] SCRIPT_NAME: `/identity` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:50:34.896899 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cba9fc-2ec7-4ca3-97d5-5050f19a7781 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80119) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:50:34.898751 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.fernet_utils [None req-30cba9fc-2ec7-4ca3-97d5-5050f19a7781 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80119) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:50:34.910442 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30cba9fc-2ec7-4ca3-97d5-5050f19a7781 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:50:34.912055 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30cba9fc-2ec7-4ca3-97d5-5050f19a7781 service placement] RBAC: Authorization granted {{(pid=80119) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:50:34.921900 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: [pid: 80119|app: 0|req: 1455/2905] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Jun 15 10:50:34 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:50:44.917614 dv-cinder-843493-2-iscsi devstack@keystone.service[80119]: DEBUG dbcounter [-] [80119] Writing DB stats keystone:SELECT=58 {{(pid=80119) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}